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1.0  INTRODUCTION 


1.1  The  Problem.:  Displays  for  Intelligent;  jy 5 tp-ms 

Guidelines  for  Che  human  factors  engineering  of  the  asm -machine  Inter  foc-fc  have 
traditionally  focused  on  sensing  end  acting:  i  .  e  .  „  display  features  and  Input 
■devices  that  conform  to  hunan,  pure eptual/mo cor  capabilities  and  preferences. 

In  recent  years,  however,  artificial  intelligence  (AI)  techniques  have  intro¬ 
duced  a  new  class  of  systems  with  which  humans  arc  required  to  interact:  sys¬ 
tems.  which  attempt  tc  replitate ,  or  improve  on,  human  reasoning.  As  intel¬ 
ligent  systems  are  proposed  for  an  expanding  sphere  of  operational  tales K  at¬ 
tention  has  begun  to  turn  to  machine  -assisted  thought,  and  to  the  manner  in 
which  computer- implemented  storage  and  transformation  of  information  tan  he 
optimally  Interfaced  with  human  knowledge  representations  and  processing 
strategies.  Human - compute r  interface  design  has  become  cognitive.  This 
report  is  intended  as  a  contribution  to  the  eirergir.g  application  of  cognitive 
science  to  human - conputE r  interaction. 

Nowhere  Is  the  challenge  greater  than  in  tba  design  of  pilot  displays  for  in¬ 
telligent  avionics  in  high-performance  combat  aircraft.  Near- future  air  war¬ 
fare  environttETits  will  be  characterised  by  increasing  aircraft  velocities,  by 
increasing  sensor  and  weapon  ranges,  and  by  increasingly  well -hidden  threats 
on  the  ground  and  in  the  air.  The  result  is  both  reduced  response  time  for 
pilots  and  heightened,  uncertainty  under  which  such  responses  must  be  made. 
Increasing  automation  of  more  routine  system  functions  (such  as  aircraft  con¬ 
trol,  target  detecei.cn,  tracking,  and  weapons  control)  has  cade  cognitive  ac¬ 
tivity,  such  as  resolving  uncertainty  and  balancing  risks,  a  relatively  more 
important  and  time -critical  component  of  the  pilot's  task.  The  natural  result 
has  been  increasing  interest  in  the  development  of  intelligent  computerized 
support  for  high-level  pilot  decisions. 

The  Interface  problem  for  such  systems  Is  formidable,  To  work  effectively, 
they  must  produce  collaborative  outputs  that  tap  potential  contributions  of 
both  human  and  computer  within  a  period  typically  of  a  few  seconds.  In  short, 
they  must  achieve  a  degree  of  cognitive  integration  of  user  and  system  that  is 
virtually  unheard  of  in  other  applications. 
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Traditional  approaches  to  the  human.' computer  iut*r£it*  (s,  g- -  summer its d  in 
Ramsey  and  ftewcad,  1979;  EhEt.il  and  Crania,  1975}  have  not  adequately  addressed 
this  problem,  For  example,  principles  for  the  design  and  formatting  of  dis¬ 
plays  are  Inadequate  for  the  portrayal  of  abstract  concepts,  such  as  threat 
value*  and  uncertainties  regarding  threat  location  and  identity,  on  which  tac¬ 
tical  dee  ie  tons  (whether  human  or  me  chine  >  must  be  based.  Similarly,  tradi¬ 
tional  guidelines  for  data  entry  are  largely  irrelevant  for  ensuring  effective 
util  tzar  ton  of  on -the -Spot  insights  by  user*  in  a  real -time  proc  n^-fi .  Artifi¬ 
cial  intelligence  contributions  to  the  user- computer  interface  have  focused 
for  the  most  part  on  input-output,  tools  (e.g.,  spatial  data  uanagenent , 
natural  language  understanding (  voice  I/O) ,  rather  than  the  effective  use  of 
those  tools  in  collaborative  human- computet  problem-solving,  Even  wort  in  the 
expert  systems  oreu  (e.g,,  on  explanation  and  mixed -initiative  dialogues)  Has 
emphasized  an  essentially  passive  role  for  users,  as  initiators  of  queries, 
recipients  of  answers  and  explanations ,  and  providers  of  raw,  undigested  data. 
One  result  has  been  the  prevalent  assumption  that  successful  real- tine  tacti¬ 
cal  systems  must  entrust  their  duties  almost  wholly  to  the  computer  and  leave 
little  or  no  opportunity  for  human  contributions.  Collaborative  aids  that 
interweave  human  and  computer  reasoning  and  decision  processes  have  evolved 
(if  at  all)  by  trial  and  error, 

Efforts  to  -develop  a  truly  cognitive  approach  to  interface  design  are,  as  yet, 
only  incipient  (cf.,  Norman  and  Draper,  l^SS) .  The  enterprise  is  difficult 
for  two  reasons  (at.  least).  First,  because  cognitive  science  is  itself  not 
yet  a  nature  discipline,  A  variety  of  models  of  human  knowledge  repre¬ 
sentation  have  been  proposed,  which  differ  in  basic  units  (e.g.+  rules,  ob¬ 
jects,  activities),  in  the  processes  that  manipulate  those  units,  and  in  the 
psychological  function's  they  are  thought  to  serve.  Second,  because  the  main 
focus  is  toward  theory  rather  than  application,  the  impl teat ions  (If  any)  of  a 
particular  cognitive  theory  for  the  design  of  an  interactive  interface  are  of¬ 
ten  far  froEfc  obvious,  Research  on  die  '’application"  of  cognitive  theories, 
therefore,  is  not  a  simple  matter  of  converting  first  principles  Into  en¬ 
gineering  diagrams.  It  must  itself  proceed  in  a  tentative,  hypo thesis- testing 
mod*,  First,  concepts  from  ba±tc  research  must  be  selected,  based  on  their  ap¬ 
parent  relevance  to  the  problem,  domain  and  empirical  plausibility;  then,  the 


implications  of  these  concepts  for  display  design  must  bo  raade  ■explicit; 
finally,  the  displays  baaed  on.  these  concepts  must  be  carefully  evaluated, 

The  results  r  in  turn,  might  provide  valuable  feedback  and  sharpened  focus  for 
basic  research. 

Our  hypothesis  is  that  recent  work  iu  cognitive  science  ecu  provide  the  under¬ 
pinnings  for  a  -pew  methodology  of  interface  design,  for  real-time  interactive 
aids.  Specifically,  that  methodology  is  based  on  insights  from  (a)  work  on 
knci-'ledge  representation  and  fb>  research  on  psychological  decision  theory', 
These  sources  are  coc.plemcntary ,  Displays  which  represent  information  in  ac¬ 
cordance  with  users1,  own  internal  teprasentfltions  should  mere  readily  util¬ 
ized,  should  b«  understood  sad  re  quickly  and  accurately  ,  end  .diDuld  provide  a 
more  effective  concept  for  eliciting  on-the-spot  user  knowledge  r  On  the 
other  hand,  human  knowledge  representations  and  information  processing 
strategies  are  imperfect;  the  Literature  on  psychological  decision  behavior 
reveals  a  lumber  of  -vJ*y*  In  which  preferred  methods  for  reasoning  may  lead  to 
biases  or  fallacies ,  Our  aim,  then,  is  to  articulate  a  design  methodology 
which  emphasizes  both  compatibility  with  user -preferred  methods  for  repre¬ 
senting  and  using  knowledge,,  and  techniques  for  avoiding  the  biases  to  which 
those  methods  ordinarily  Lead. 

1.2  Ob' actives  and  Scone 

The  research  reported  here  is  the  product  of  a  £■ months  Phase  I  effort  in  the 
Small  Business  Innovative  P^e search  (SBIR)  program.  The  objectives  were  Co; 

■al  examine  relevant  theories  and  concepts  from  research  on  knowledge 
representation,  behavioral  decision  making <  and  decision  aiding, 

b)  develop  a  cf.ethodjology  for  generating  display  design  concepts  for 
pilot  interaction  with  intelligent  systems,  based  upon  those 
theories , 

c)  use  tbo  inethodrvlogy  to  develop  experimental  display  design  concepts. 
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d)  conduct  preliminary  feasibility  tests  of  those  concepts. 


The  initial  application  context  involved  an  air-to-ground  strike  mission.  In 
order  to  reach  a  target  deep  within  enemy  territory,  an  aircraft  must  avoid  or 
defeat  a  variety  of  surface  threats  whose  Identity,  location,  and/or 
capabilities  may  be  wholly  or  partly  unknown.  Information  may  be  obtained 
during  the  flight  itself  from  on-board  sensors  or  radio  messages  from  air  or 
ground  stations  which  in  some  cases  can  help  identify  new  threats  „  resolve  the 
uncertainties  in  prior  intelligence,  and  help  pilot*  seleoc  .in  adaptive 
response  {erg,r„  a  revised  potict)  „  Several  overlapping  and  interne  laired  topic? 
were  of  specific  initial  concern  to  us  within  this  content; 

o  dynamic  displays,  that  is,  displays  that  change  as  the  mission 

progresses,  as  new  threat  information  is  received,  or  as  computa¬ 
tions  modify  conclusions  about  threat  assessments  of  preferred 
poutt-s  and  tactics; 

o  Lufcerrauttcy ,  how  pilots  think  about  it,  how  it  affects  their  deci¬ 

sions,  and  how  displays  should  be  designed  to  represent  it; 

o  hierarchically  organized  inf onpa t i on ,  i.e.,  how  information  should 
be  aggregated  so  that  displays  arc  uncluttered  and  the  pilot's  at¬ 
tention  la  focused  on  the  appropriate  level  o£  detail; 

o  eaplAOatfert*  of  system  rd^rortJh#,  l,e.„  how  to  display  in  a  clearly 
intelligible  way  th*  basi*  of  inf * repot*  from  incomplete  and  unreli¬ 
able  data  and  the  reasons  for  recommended  course*  of  action  within 
the  limited  available  response  time. 

fhase  I  specifically  excluded  consideration  of  air  throats  and  air-tc^air  mis¬ 
sions  .  Further,  the  principle  focus  was  on  in-flight  pilot  aids  as  opposed  to 
prestrike  ground  planning.  Never the Less  r  certain  aspects  of  the  planning 
process  had  Co  be  considered,  specifically  the  role  of  intelligence  informa¬ 
tion  and  uncertainty  about  threat  la cat! on  and  type,  in  order  to  understand 
the  inpact  of  new  information  received  during  the  mission. 
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Finally,  the  emphasis  in  this  phase  af  the  study  was  on  the  development  of  a, 
methodology  based  upon  the  •underlying  relevant  theory  j  and  on  display  concepts 
that  illustrated  the  application  of  this  theory,  rather  chan  on  ebe 
development  of  detailed  prototype  displays-  Thus,  In  balancing  the  amount  of 
effort  to  be  deveted  during  Phase  1  on  theory  and  method  versus  software 
development,  the  emphasis  was  on  theory  and  method, 

1 . 3  Approach  end  Ovqt-v  l  ew  of  the  Report 

The  approach  consisted  of  several  steps: 

a)  A  critical  review  was  conducted  of  the  research  literature  dealing 
with  knowledge  representation  (especially  mental  models)  find  be¬ 
havioral  decision  theory  (especially  the  wort  on  cognitive  biases 
leading  to  errors  ip  Judgment)  in  order  to  identify  relevant 
theoretical  formulations  for  an.  in-flight  pilot  display  design 
methodology „ 

b)  Structured  interviews  were  held  with  three  experienced  Air  Force 
pilots  of  tactical  strike  aircraft,  in  which  they  were  led  through  a 
typical  mission,  new  threat  information  was  presented  periodically, 
and  they  were  asked  how  they  thought  about  the  situation  as  it 
developed,  the  uncertainties  inherent  in  the  situation  assessment, 
and.  the  choice  of  responses.  Questions  were  designed  to  probe  their 
ways  of  mentally  orga.nl  £in.g  and  representing  information.,  potential 
biases  in  making  decision!; ,  and  the  type  of  displayed  Inforwation 
and  method  of  display  that  would  most  help  them  in  handling  uncer¬ 
tainty  end  reaching  a  timely  decisfon. 

c)  The  design  methodology  was  applied  to  data  elicited  from  the  pilots, 
snd  a  series  of  preliminary  pilot  displays  was  developed.  The 
preliminary  displays  were  programmed  on  an  IfiK-FC/AT  In  a  sequence- 
keyed  to  a  ml  as i on  scenario.  These  displays  conformed  to  the  con¬ 
straints  Imposed  by  mental  model  theory,  while  providing  prescrip¬ 
tive  guidance  based  on  behavioral  decision  theory. 
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d)  The  demonstration  system,  was  reviewed  individually  by  the  three 

pilots  who  had  bean  interviewed  initially,  Ratings  were  solicited 
from  the  pilots  rcgeedln.g  specific  features  on  each  display,  The 
ratings  wcKt  based  on  a  7 -point  stale  from  1  (very  .good)  through  4 
(neutral)  tn  7  (very  had),  and  comments  were  solicited  to  explain 
the  reasons  underlying  the  ratings  and  to  suggest  improvements  or 
alternative  designs.  During  this  review  (which  was  tape- recorded) , 
the  research  tua=.  attempted  to  further  clarify  the  mental  mode is  and 
decision  strategies  underlying  the  pilots'  responses. 

ft)  Finally  a  demonstration  version  of  the  final  display  concepts  was 
developed  and  demonstrated  at  Wright -Patterson  Air  Forte  Ease. 

In  Section  2,0  below  »e  examine  the  relevant  cognitive  science  literature  and 
describe  a  methodology  for  the  design  of  displays  for  intelligent  systems. 
Section  J,0  then  presents  the  results  of  applying  that  methodology  to  the 
preliminary  design  and  evaluation  of  in-flight  pilot  displays.  Finally,  Set- 
cioia  4,0  summarises  the  conclusion?  from  Ph^se  I  and  poinrs  coward  future  re  ■ 
search . 
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2.0  COGNITIVE  SCIENCE  FOUNDATIONS  FOR  INTERFACE  DESIGN  METHODS 


In  this  section  we  pc^dsc  a  theoretical  has  Is  for  a  methodology  oE  cognitive 
Interface  design.  As  noted  in  Section  1.1,  that  methodology  has  a  dual 
oasis;  (1)  displaying  information  In  a  way  that  is  compatible  with  a  decision 
maker's  preferred  method  of  representing  knowledge?:  and  salving  problems;  while 
{2}  providing  protective  devices  to  guard  against  associated  blasts.  Thus, 
the  two  major  areas  of  cognitive  science  research  of  concern  to  us  art  models 
of  humeri  knowledge  representation  and  reasoning,  end  research  on  errors  In 
judgment  and  decision  making,  This  by  no  means,  therefore,  purports  to  be  .i 
complete  review:  there  is  considerable  additional  cognitive  research  litera¬ 
ture  with  an  important  beating  on  human -computer  interaction.  Eta  the  r,  we 
focus  here  on  work  which,  on  the  one  hand,  hat  boon  relatively  neglected  in 
the  context  of  system  design,  and  which,  on  the  othor  hand,  has  been  the  major 
source  of  Insights  for  the  design  methodology  which  we  propose, 

k'e  argue  that  these  two  research  trod  it  ions  are  complementary  ant!  con  shod 
light  on  one  another  both  at  a  theoretical  level  and  in  thoir  application  eo 
design.  Sections  2.1  through  2.3  examine  this  literature,  while  Section  2-A 
extracts  their  implications  for  display  design. 

2.1  levels  of  C.PJUiitiYe„FerfcrmaTiC& 

Rasmussen  (19&3;  19B6}  has  introduced  a  classification  of  levels  of  human  per¬ 
formance  which  will  serve  as  a  useful  starting  point  for  the  knowledge  repre¬ 
sentation  concepts  to  be  developed  in  the  next  section.  As  shown  in  Figure 
2-1,  Rasmus-ten  distinguishes  performance  which  is  skill -Wind ,  ml  c -based,  end 
knowledge  -  baj; cd , 

Skill -based  behavior  involve*.  smooth,  automated ,  highly  Integrated  patterns  of 
behavior  in  which  the  body  typically  acts  as  a  ’’mu]  ci variable  continuous  con¬ 
trol  system  synchronising  moveuents  with  the  behavior  of  the  environment . " 
Sensory  inputs  serve  two  functions  at  this  level:  as  “signs “  which  trigger 
appropriate  behavioral  patterns  {e.g.,  an  incoming  nissile  elicits  the 
response  pattern  of  taking  evasive  action} ;  and  as  “signals"  which  modulate 
and  control  an  already  activated  pattern  (e.g.,  observation  of  the  distance 
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Figure  2-L:  FUsmyssen' s  Framework  fOF  Cognitive  Pcttor&ftrice 


and  angle  of  approach  of  the  missile) ,  Skill-based  behavior  is  not  typically 
a  matter  of  simple  feedback  control.  Rather,  it  depends  on  a  flexible  and 
dynamic  internal  model  of  the  environment ,  which  is  continually  updated  by 
signals  fror.  the  environment  ,  which  permits  the  individual  to-  anticipate, 
likely  environmental  pertw that ions ,  and  which  iflttgVittl  activities  into  a 
single,  smooth  sequenfie ,  Pilots  engaged  lr  in  evasive  maneuver,  for  example, 
may  ha^o  instant  tbr£o- dimensional  mental  "picture1*  of  the  entire  pattern 
to  be  executed  by  the  aircraft  and  an  automated  unconscious  set  of  behavioral 
routines  for  carrying  it  out. 

Ac  cho  next  hlghut  level  of  performance*  rule -based  behavior  is  consciously 
controlled  by  a  stored  rule  -or  procedure.  Such  a  nule  may  have  been  acquired 
by  direct  experience  or  it  may  have  been  learned  from  other  people  by  instruc¬ 
tion.  For  example,  a  pilot  may  discover  the  app mpr lo to  distenco  and  altitude 
for  avoiding  detection  by  a  particular  enemy  missile  site  through  his  own  ex¬ 
perience  {e.g.,,  of  being  illuminated  by  its  tracking  radar  at  certain  loca¬ 
tions  and  not  at  cther-S)  ;  or  he  taay  have  been  briefed  on  what  to  do  in  the 
vicinity  of  such  a  threat  during  mission  planning.  Rule -based  behavior  is 
goal -oriented  only  in  a  limited  sense:  behavior  is  governed  by  rules  that 
wore  successful  in  previous  performance  (one's  own  or  others'!.  Rut.  the  goal 
remains  implicit  in  the  use  of  the  rule"  there  is  no  explicit  reasoning  ox 
p fob 1 era- solving  to  discover  the  best  way  to  achieve  the  goal.  Individuals  may 
acquire  a  large  store  of  rules  at  this  level  which  enable  them  to  respond 
adaptively  to  relatively  familiar  or  expected  situations. 

The  next  level  of  performance,  knowledge -baaed  behavior,  is  relevant  when  the 
situation  is  unfamiliar.  If  no  rules  are  available  for  achieving  the  goal, 
the  individual  must  draw  upon  a  deeper  understanding  of  the  causal  relaticn-- 
ships  in  the  environment  which  determine  the  conditions  under  which  his  goal 
can  and  cannot  be  achieved.  He  must  construct  a  "mental  model"'  of  the  situa¬ 
tion  in  which  alternative  courses  of  action  and  alternative  outcomes  can  be 
simulated.  Fox  example,  a  pilot  confronted  by  conflicting  information  about 
the  classification  or  location  of  a  threat  c.sy  utilise  his  understanding  of 
the  strengths  and  weaknesses  of  the  different  sources  of  information  under 
various  conditions  to  resolve  the  conflict.  Confronted  by  an  unusual  con¬ 
figuration  of  unexpected  threats  on  his  flight  path,  he  may  utilise  his 
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knowledge  of  threat  eapabili ties  and  tactics  to  mentally  "simulate"  alterna¬ 
tive  routes ,  At  this  level,  errviten.nieio.tal  inputs  no  longer  function  as 
"signs”  which  are  associated  with,  p  re  Learned  procedures  ,  but  as  "'symbols11 
which  provide  evidence  for  functional  properties  and  causal  relationships. 

The  development  of  cognitive  capabilities  often  involves  transfer  *f  control 
from  a  higher  to  a  lowet  Level.  Thus,  an  Initial  Ltigo  of  role- following 
(e.g.r  relying  oft  instructor  3ftd:  tsKtbocJc  in  the  operation  of  a  flight 
simulator)  will  be  replaced  after  ft  period  of  direct  practice  by  a  more 
automated  and  intuitive  node  of  operation.  Similarly,  basic  knowledge  of  the 
causal  and  functional  properties  of  a  domain  (e . g. ,  characteristics  of  weapons 
and  threats)  will  he  replaced  after  experience  with  mere  stereotyped  rule- 
based  react Lons.  Hove rthe less,  higher- level  control  may  occasionally  inter¬ 
vene  In  the  execution  of  a  well- practiced  capability.  Rules  cay  control  nht 
sequencing  of  shilled  routines  or  impose  constraints  on  how  the  skill  is  ex¬ 
ecuted  (e,g.,  “the  incoming  missile  is  a  very  fast  one,  sc  execute  the  evasive 
pattern  quickly. “I  Similarly,  when  rules  prove  inadequate  for  performance  in 
novel  situations  £e.g,f  conflicting  evidence  or  unexpected  threats),  it  is 
necessary  to  ascend  to  higher  level  knowledge -based  reasoning  In  order  to 
determine  what  to  do  and.  perhaps,  ce  generate  appropriate  rules, 

2 . 2  Kr.owl  edge  Pep  r r  r.  e  ft tatl  ons 

Improved  understanding  of  pilot  cognitive  processes  can  be  obtained  by  going 
beyond  Rasausjien'  s  scheme  ce  a  more  detailed  consideration  of  the  knowledge 
representation*  required  to  implement  it:  fltit,  by  introducing  a  more  active 
and  hierarchical  representation  of  stereotypical  information  at  the  *rule- 
besed"  level:  and  second,  by  examining  constraints  on  performance  derived  from 
the  nature  and  function  of  mftncai  models, 

2.2.1  Schemas  end  scripts.  In  his  discussion  of  performance  in  familiar 
situations,  Rasmussen  imp lies  that  knowledge  of  this  type  is  composed  of  Small 
unrelated  units  (rules)  which  are  activated  in  a  stimulus -driven,  or  "bottom 
up,"  fashion,  An  example  of  this  type  of  control  iE  a  standard  production 
system  which  contains  a  large  number  of  rules  of  tbs  form  "If  <£ituatiori>  then 
<action>.  11  When  the  conditions  specified  in  the  antecedent  of  the  rule  are 
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iflttsfied.  the  action  described  in  the  consequent  is  patterned-  That  action 
may  create  conditions  which  cause  ether  rules  Co  fire,  and  so  an, 

A  large  body  of  cognitive  science  research  suggests,  however,  Chat  human  per¬ 
formance  even  in  familiar,  stereotypical  situations  involves  more  highly 
structured  types  of  knowledge  and  slots  active,  "top-down"  processing  chan  are 
found  in  the  standard  production  system.  The  notion  of  a  schema  (or  frame) 
provides  a  convenient  means  of  representing  knowledge  of  this  type.  Schemas 
arc  data  structures  corresponding  to  familiar  types  of  objects,  situations, 
events,  sequences  of  events,  actions,  or  sequences  of  actions  (Euraelhatt  and 
Butman.  19&5)  . 

Three  features  of  schema-based  representations  are  General; 

(1)  Schemas  have  slots  or  variables  which  specify  which  types  of  infor¬ 
mation  it  is  appropriate  to  seek  about  a  particular  type  of  thing, 
For  eK ample ,  a  pilot's  schema  for  a  surface  ■  to -air  missile  site 
might  include  slots  for  radar  range ,  radar  altitude,  missile  range , 
missile  effectiveness ,  local  terrain  features ,  etc,  In  some  cases 
sloes  may  have  default  values ,  t.v.,  values  which  are  expected  or 
assumed  to  be  correct  until  evidence  to  the  contrary  la  obtained. 

For  example.,  pilots  may  cautiously  assume  that  a  missile  of  unknown 
type  has  maximum  capability  until  they  learn  otherwise . 

{2)  Schemas  represent  knowledge  at  multiple  levels,  and  these  levels  axe 
hierarchically  organa z ed,  both  in,  terms  of  "is -a -part - o£lf  relation¬ 
ships  and  in  terms  of  "  is -a-kind-of"  relationships  Schemas  typi¬ 
cally  include  other  schemes  as  parts:  e.g,,  the  SAM  site  schema  may 
include  sub -schemas  fot  each  major  component  of  the  site  (radar, 
missile,  terrain).  In  addition,  schemes  may  exist  fox  types  of  ob¬ 
jects,  events,  etc,  at  varying  levels  of  generality i  for  example, 
there  may  be  a  general  schema  for  weapons,  a  more  specific  schema 
for  anti-sir  weapons,  a  still  more  specific  schema  fer  surface-to- 
air  missiles,  and  a  schema  for  a  specific  type  of  surface-to-air 
missile  (e.g.,  SA-2) -  Each  schema  inherits  slots  and  default  value? 
from  schemas  above  it  in  the  generalisation  hierarchy.  Both  types 
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of  hierarchical  organization  provide  powerful  tools  for  generating 
expat tat ions  and  guiding  the  collection  of  now  information- 

(3)  Finally,  schemas  arc  active  processors  of  Information,  father  than 
static  repositories  of  facts-  The  ensemble  of  s cheats  embodying  a 
person's  knowledge  works  together  to  makfc  sense  of  incoming  data  and 
to  guide  action-  Each  schema  continually  assesses  its  own  up* 
plitabillty  to  the  current  situation,  determines  what  further  Infor¬ 
mation  should  be  sought  or  expected,  and  forwards  relevant  findings 
to  ocher  schemas  (cf  r ,  Kinsley,  1979)  .  Schamas  thus  bridge  the 
traditional  gap  between  static  "declarative"  represents! i on$  and  ac¬ 
tive  "procedural"  representations  ('m-t  nograd ,  1972), 

The  pattern  of  cormnun (cation  among  schemas  may  be  determined  jointly  by  ebu 
task  and  by  their  hierarchical  organization.  In  inference  tasks  like  diag¬ 
nosis  or  classification,  a  relatively  generic  schema  (eg. ,  an  antiair  missile 
site)  may  respond  to  available  evidence  by  deciding  that  it  applies  to  a 
situation  and  then  activate  schemas  for  specific  subtypes  (e.g.f  SA-2  ,  3A- 8 , 
SA- 9  ,.,):;  the  subtypes  then  compete  to  determine  which  of  them  applies;  and. 
so  on  down  a  hierarchies!  tree  (of.,  Chandzasekaran ,  1993).  The  task  of  plan¬ 
ning  may  also  involve  increasingly  detailed  specification  of  schemas.  But 
planning  rosy  also  involve  the  schema  for  part  of  an  activity  activating  the 
schema  for  the  whole,  which  in  turn  activates  schemas  for  other  (subsequent) 
parts  of  the  activity.  Schemas  for  activities  thus  not  only  support  the 
process  of  recognizing  or  interpreting  what  is  going  on,  but  once  the  context 
has  been  recognized,  determine  what  actions  an  agent  should  take  within  it 
(GaLembos,  Abe Leon  and  Black,  1986), 

A  specific  type  of  schema,  suited  for  representing  knowledge  about  familiar 
activities,  is  the  script  (Sehank  and  Abe Is on,  1977),  A  script  contains  slots 
whose  values  specify  the  objects  {" props’)  and  persons  ("roles")  which  par¬ 
ticipate  in  the  activity,  its  entry  conditions,  and  results,  as  well  as  the 
sequence  of  teeitK  which  constitutes  the  activity.  For  example.  Figure  2-2  is 
a  hypothetical  pilot'  s  script  for  an  offensive  conn ter- air  (OCA)  mission,  and 
contains  slots  far  the  aircraft,  target,  IF,  way -points,  and  mission  com¬ 
ponents  (or  scenes) .  Such  a  script  encapsulates  the  pilot's  pres tore d 
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Figure  2-2;  An  Illustrative 


Script  Representation  f&r  an  OCA  Hiss ten 


knHjwLui%ei  about  OP  A  missions;  dnd  guides  ht-=  eXpe  ctati  ons  and  act  tans  as;  he 
proceeds.  (In  a  real  script,  scenes  would  be  specified  in  much  greater 
detail. > 

Both  types  of  hierarchical  organization  ( is -a-part- of  and  ia ■ a- kind- of >  are 
relevant  in  script-based  representations.  First,  scripts  may  be  hierarchi¬ 
cally  composed  of  scones  <e . ,  ingress,  attack,  egress >  which  are  th*ms»lv*s 
script*  or  which  contain  scripts  as  parts.  Second,  in  recent  worlk  Schenk 
(19£2)  has  described  no re  general  schemas  (called  MOPS)  of  which  scripts  are 
instances.  Planning  nay  involve,  activation  of  relevant  generic  schemas  fol¬ 
lowed  by  a  process  cf  filling  in  details  until  a  specific  script  Is  con¬ 
structed  (cf.  ,  Stefik,  1981).  Foe  example,  pilots  may  have  generic  schcaas 
for  strike  missions  which  determine  son*  of  cbe  features  of  OCA-nissioa 
scripts.  On  an  even  more  general  Level,  a  schema  cal  lad  5JM- Performance 
provides  an  abstract  characc-erizati on  which  applies  to  OCA-missions  as  well  as 
to  any  other  "perf orioa-nce . H  According  to  Schar.k,  this  schema  contains  eight 
universal  scenes;  Preparatory  (things  done  prior  to  entering  a  content) , 
Enablement  (entry  into  a  content) ,  Fre- Condition  (things  done  prior  to  the 
main  activity).  Side -condition  (tangential  actions),  Action  (the  main, 
activity),  Post- condition  (tying  up  loose  ends),  Disenab  lenient  (leave),  anc 
Transition  (move  on  to  nsw  things) .  Figure  2-2  illustrates  how  scenes  in  the 
OCA  mission  script  might  be  organised  under  these  categories 

The  concept  of  a  script  provides  a  considerably  richer  and  mote  adequate  rep¬ 
resentation  of  stereotypical  performance  than  the  notion  of  a  rule.  ScrlpCs 
in  fact  provide  a  unifying  content  for  other  types  of  knowledge- -both  in  the 
form  of  rules  (at  the  mc-st  specific  level  within  a  scene)  end  in  the  form  of 
other  types  of  schemas  (e.g.,  about  threats,  the  aircraft,  and  terrain 
features),  by  shoving  where  and  hov  they  become  relevant  in  the  course  of  a. 
familiar  activity  (Letfdo,  Hu.1  1  in ,  and  Cohen ♦  1967). 

2.2,2  Bengal  jpode^s.  Scripts,  by  definition,  provide  no  capability  for  deal¬ 
ing  with  novel  or  unexpected  situations,  When  deviation;;  from,  a  script  occur, 
or  no  familiar  script  can  be  found  which  adequately  watches  the  given  cir¬ 
cumstances,  knowledge -based  reasoning  must  he  invoked,  Mental  modela  may  be 
employed  to  explain  apparently  anomalous  events  or  to  generate  options  that 
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overcome  unanticipated  obstacles  .  Given  the  severe  tine  pressures  constrain- 
ln£  pilot  performance „  it  is  unlikely  chat  they  are  able  to  nake  frequent  ef¬ 
fective  use  of  mental  aodels  in  this  way.  Nevertheless,  the  potential  con¬ 
tribution  of  pilot  knowledge  at  this  level  is  great,  as  is  the  need  for  hum^n- 
computer  systems  that  can  cooperatively  adjust  to  unexpected  circumstances,  A 
r.ajor  function  of  an  intelligent  avionics  system  may  be  to  automate  relatively 
routine  or  stereotypical  tasks,  and  to  alert  pilots  when  high-level  "manager¬ 
ial"  or  11  troubleshooting"  skills  are  required  (c£.  .  Hose,  Seising,,  and  Hudson, 
193£t),.  Thus,  it  will  be  worthwhile  to  explore  theories  about  the  way  people 
naturally  solve  problems  at  this  Level. 

Vhat  is  a  mental  model?  A.  variety  of  reviews  and  taxonomies  of  this  concept 
now  exist.  Rouse  and.  Morris  (19B6) f  building  on  Rasmussen  (1979),  provide  a 
functional  definition  of  mental  models  as  "the  mechanisms,  wbettby  humens  are 
able  to  generate  descriptions  of  system  putpoi*  and  form,  explanation*  of  Sys¬ 
tem  functioning  and  observed  system  states,  and  predictions  of  future  *y* tern 
states , "  They  then  discriminate  among  mental  models  in  different  domains 
( e .  g , problem  solving  in  physic*  versa*  manual  control)  based  on  whether  or 
not  a  person  is  aware  of  his  or  bar  manipulation  of  a  mental  model  and  the  ex¬ 
tent  to  which  use  of  the  model  i*  a  par ter  of  choice  as  opposed  to  being  dic¬ 
tated  by  the  task,  Yeung  (19 £3)  ettmaerates  a  variety  of  mental  model 
mechanisms  that  have  been  proposed  in  the  literature  (e.g..,  analogy,  device 
surrogate,  mapping,  problem  space,  grammar) r 

These  di*cu*sfor£  fail  to  provide  u  basis  for  the  mental  model  concept  which 
links  the  proposed  properties  of  mental  models  {e.g.^  as  reviewed  by  Young)  to 
the  functions  they  are  meant  to  perform  (e.g.,  as  described  by  RssixuEseu  and 
Rouse  and  Hurt is) .  An  understanding  of  that  linkage  is  required  in  order  to 
sett  out  and  evaluate  diverse  definitions  and  theories.  Moreover,  Rouse  and 
Morris  do  not  distinguish  between  knowledge  that  is  simply  retrieved  by  tneens 
of  a  mental  model  and  knowledge  which  is  generated  for  the  first  time  by  such 
models.  We  would  argue  that  a  theory  of  mental  models  must  in  fact  make  this 
discrimination;  that  it  should  begin  with  the  function  of  generating  new 
knowledge  and  the  constraints  that  function  imposes  on  representational 
properties;  and  that  the  use  of  mental  models  to  support  stereotypical  skill 
in  familiar  situations  is  derived  from  this  more  basic  function. 
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In.  his  classic  book.  The  Sciences  of  the  Artificial ,  Herbert  Simon  (.1969} 
argues  that  "human  problem  solving,  from  the  most  blundering  to  the  most  in¬ 
sightful,  involves  nothing  mere  than  varying  mixtures  o£  trial  and  error  -and 
selectivity.  The  selectivity  derives  from  various  rule$  of  thuith,  nr  heuris¬ 
tics,  that  suggest  which  paths  should  be  tried  first  and  which  leads  arc 
promising"  (p.  97),  Within  our  framework,.  Simon'?  ■ selectivity1'  cort^s ponds 
to  stereotypical ,  pre-existing  knowledge,  and  "trial  and  error"  corresponds  co 
knowledge -bused  reasoning,.  As  Simon,  noces,  "the  more  difficult  and  novel  the 
problem,  the  greater  is  likely  to  be  the  amount  of  trial  and  error  required  to 
find  a  solution"  (p.95). 

In  a  recent  discussion,  D.C,  Dennett  (1979)  has  argued  that  the  prominence  of 
generate -and- test  (or  '"trial  and  error"')  3fiC:ohantSc.s  in  A,I  programs  is  no  acci¬ 
dent,  that  any  process  in  which  genuinely  new  knowledge  is  created  within  a 
system  must  involve  soma  version  of  variation  (trial)  and  selection  (error). 
First,  a  inc  e  the  knowledge  i  s  new ,  it  must  b  e  unde  rd<?  termine  d  by  the  pre  - 
existing  design  (i.e. ,  knowledge)  of  the  system ^  Its  other  wordi ,  there  must  be 
a  process  off  variation  or  option  generation  that  Is  to  some  degree  random  or 
fortuitous.  Variation  by  itself,  however,  nan  provide  no  more  than  a  chance 
probability  of  improving  on  the  old  design.  If  the  products  of  random  varia¬ 
tion  are  to  produce  new  knowledge,  there  must  he  some  process  oe  selection 
which  can  reject  variations  on  the  basis  of  what  is  previously  known. 

Generate- aud- test  mechanises  operate  at  a  variety  of  levels,  which  vary  In  the 
degree  to  which  the  processes  of  selection  take  place  within  the  o r ganism .  In 
natural  selection,  variation  in  the  genetic  code  nay  produce  novel  behavioral 
dispositions,  percep tual/mb r.or  skills,  ate.  The  selection  process  is  cot  In¬ 
side  the  organism  at  alii  rather,,  it  work*  through  the  differential  survival 
of  o  rgan Isms  in  vhi th  such  va  ri ations  turn  out  to  be  envi ronment  ally  adap  t Ive „ 
Gene  rate -and- test  mechanisms  are  also  essential  to  learning  at  the  individual 
level.  'Variations  in  an  individual's  behavior  will  bn  retained  and  reoccur 
whan  they  produce  environmental  consequences  which  are  perceived  as  rewarding 
(or  prevent  environmental  events  perceived  as  aversive) .  The  selective  events 
(the  perception  of  reward  or  pain)  are  now  Inside  the  organism,  but  the  en¬ 
vironment  still  controls  vJren  they  occur,  by  causally  linking  them  to 


behavioral  vari  at  ions  r  Know) edge -based  reasoning  carries  tbs  gene  rate-  and- 
test  concept  one  step  further,  entirely  internalizing  the  selective  process; 
hypotheses  are  varied  in  an  internal  model  of  the  environment  ,  with  selective 
retention  of  alternatives  that  prove  successful  inside  that  internal  model. 

The  effectiveness  of  learning  in  creating  genuinely  new  knovlc-dgo  depends  on 
two  things ;  {a)  the  independence  (!.<.,  'randomness" )  of  the  generation,  func¬ 

tion  {which  is  within  the  organism)  with  respect  to  the  selection  function 
(which  is  controlled  by  the  environment) ;  and  (b)  the  selection  process 
replicating  to  a  reasonable  approximation  the  affects  of  evolutionary 
selection;  i , a , ,  the  pleasures  and  pains  that  shape  an  individual's  behavior 
should  be  correlated  to  a  degree  with  ultimate  reproductive  success  and 
failure.  That  the  latter  is  the  case  is  largely  ensured  by  the  fact  that  the 
capacity  for  learning,  and  the  particular  events  that  serve  as  positive  and 
negative  consequences .  have  themselves  evolved  through  natural  selection. 
Similar  conditions  must  apply  for  know)  edge -based  reasoning  to  be  an  effective 
method  of  creating  new  knowledge,  Tb*  variation  function  within  the  organism 
must  be  independent  (i.e,,  "random")  with  respect  to  the  selection  function 
since  otherwise  we  have  pre-existing  (stereotypical)  knowledge;  and  the  selec¬ 
tive  function,  also  within  the  organism,  ciust  replicate  (.reasonably  well)  the 
selective  action  of  the  env ironnent  in  learning- -1 , e , ,  it  must  produce  inter¬ 
nal  selective  events  In  the  saa?  causally  cpproprlace  way  a*  the  environment 
does,  At  first  glance,  it  is  a  mystery  how  this  could  be;  either  the 
knowledge  that  a  particular  variation  produces  a  particular  selective  effect 
is  already  present  in  the  organism  (and  so  genuinely  new  knowledge  is  not 
produced)  or  the  knowledge  is  not  present  (and  intelligent  selection  is  not 
possible) . 

At  the  knowledge - b as ed  level,  then,  variation,  and  selection  must  at  the  same 
time  be  uncoupled  within  the  organism  (to  achieve  randomness  of  variations) 
and  in  another  sense  coup Led  (so  that  the  selective  function  can  "know*  which 
variations  arc  likely  to  be  adaptive).  Saver  ill  important  representational 
properties  of  mental  cWcl*  ate.  suggested  by  th*  requirement  that  both  of 
tfcieso  conditions  be  simultaneously  satisfied; 


a  presence  in  the  Bedel  of  component (s)  in  which  variations  are  repre¬ 
sented  (e , g ,  ,  actions  which  are  hypothesized  to  achieve  an  objective 
or  states  of  affairs  which  are  hypothesized  to  account  for  unex¬ 
pected  or  anomalous  events); 

o  presence  in  the  model  of  c opponent (s)  in  which  success  or  failure  In 
achieving  some  select ive  criterion,  is  represented  (e.g.,  achievement 
of  action  goals ;  explanation  of  anomalous  or  unexpected  events); 

o  representation  of  relationships  between  variation  component (s)  and 
selection  component (a)  in  such  a  wav  that  when  changes  are  made  in 
the  variation  component (s) ,  corresponding  causally  or  logically  ap¬ 
propriate  changes  occur  in  the  selection  component (a ) i  and 

o  absence  of  a  pre-existing  direct  representation  of  these  causal  or 
logical  relations  (at  any  level  of  generality);  for  example,  no  ex¬ 
plicit  rules  of  the  form  11  If  ^variation  x>  then  <value  y  on  selec¬ 
tion  criterion^-"  or  "If  Oariation  of  type  X>  then  <vaLue  of  typo  Y 
on  selection  criterion^. " 

Detailed  Implement* ti cm  of  these  properties  mlgbc  be  accomplished  in  more  than 
one  way.  However ,  we  would  argue  that  any  successful  iup lamentation  must  in¬ 
volve  certain  common  features:  the  notion  that  a  mental  model  consists  of 
r.ultiple  eompoiients ,  that,  pre-existing  causal  or  logical  knowledge  about  its 
own  behavior  ia  associated  with  each  component,  and  that  novel  information 
about  the  adaptive  adequacy  of  random  variations  is  derived  by  ‘'gluing1  the 
components  together  and  observing  their  interaction. 

This  is  what  a  pilot,  does,  for  example,  when  in  Che  face  o-f  unexpected  threats 
ho  imagines  an  alternative  route  or  an  alternative  set  of  tactics,  (e.g.  ,  EGK, 
chaff)  and  "plays  out"  the  consequences  of  the  option  in  bis  mind:  what  will 
each  enemy  unit  think  and  do?  what  will  he  do  in  turn?  etc.  The  components 
of  the  model  are  familiar  {hie  own  aircraft  and  its  capabilities,,  the  threats 
and  their  capabilities);  but  the  configuration  is  novel r  In  order  to  evaluate 
an  action,  therefore,  he  must  put  the  components  together  and  internally 
"observe 11  their  interaction. 


A.  theory  of  aental  models  with  these  features  ha*  he  cm  developed  by  deflect 
and  Brown  {196))  .  ft  mental  otod-cl.,  am  their  view,  consist*  r  first,  o£  a 
“device  copol&gy 3 11  i,«.,  a  set  of  veil -under stood  components ,  a  sec.  of  well- 
undersccod  “conduits"  (connections  by  means  of  which  components  may  causally 
affect  cne  another)  „  and  a  specification  of  which  coup  amenta  are  connected, 
with  which  by  conduits.  Thus  each  component  has  a  set  of  states  it  can  be  in 
(e.g.,  detected  or  undetected  as  states  af  own  aircraft;  detecting  or  not 
detecting  ns  states  of  an  enemy  radar  installation) „  and  a  set  of  rules  deter¬ 
mining  how  its  state  will  change  as  a  function  of  changes  in  the  values  of 
conduit  attributes  (a.g. .  „  ECEi  or  chaff)  „ 

In  terms  of  the  previous  section,  we  can  understand  dcKleer  end  EtOWr/s  notion, 
of  a  "device  topology"  as  a  system  of  schemes  which  represent  knowledge  about 
tho  properties  end  behavior  of  objects,  and  which  s*nd  "  rae.S  sagea  "  to  one 
another  representing  cause -effect  relationships  end  triggering  thH  changes 
in  the  recipients.  A  key  feature  of  this  type  of  aodel  is  the  .louaiifcy  o£ 
these  CHUse-flff*ftt  relationships ;  that  is  ,  rules  fur  the  behavior  uf  any  given 
camp on*nt  esn  only  reference  its  own  state  and  the  attributes  af  the  conduits 
connected  to  it,  and  can  in  tin  way  refer  to  haw  the  overall  system  Is  known  or 
intended  to  function,,  For  example,  if  the  pilot  already  knows  that  if  he 
adopts  a,  certain  tactic,  he  will  not  be  detected,  there  is  no  point  in  utiliz¬ 
ing  a  mental  model,  deKIeer  and  firown  call  this  the  “no -function -in- structure 
principle,*1  and  it  represents  the  "uncoupling"  which  is  essential  for  the 
model rs  ability  to  generate  new  knowledge  from  old  knowledge.  If  mental 
models  are  to  serve  their  purpose  of  generating  predictions  in  novel  cir¬ 
cumstances,  e . g , h  about  the  outcome  of  an  option  or  the  impact  of  a  causal 
variable,  they  cannot  rely  on  prior  t stereo typical)  knowledge  of  what  is  to  be 
predicted. 

deKIeer  and  Brown's,  theory  needs  to  bo  supplemented,  however,  by  recognition 
that  prior  knowledge  of  a  ’non- loco 1"  sort  (1.®.,  know) edge  which  goes  beyond 
the  in formation  encapsulated  in  the  separate  object  schemas)  does  play  a  cru¬ 
cial  role  in  mental  models,  in  at  least  two  ways.  First,  as  Simon  {1M9> 
noted,  there  is  typically  some  selectivity  in  the  generation  of  options  for 
testing.  He  would  argue  more  strongly  that  some  Selectivity  must  always  be 


present;  otherwise,  t  lrtte  there  is  an  infinite  spite  of  potential  eolation*  to 
be  searched,,  adaptive  possibilities  would  hardly  ever  be  found,  Prior 
(stereotypical)  knowledge  supports  such  selectivity;  by  narrowing  the  field 
within  which  options  are  generated  for  testing  (e.g.f  there  are  sone  things 
the  pilot  already  knows  will  not  work) r  by  providing  components  or  building 
blocks  for  options  which  can  be  recoabined  in  novel  ways ,  or  by  bringing 
promising  possibilities  to  mind  by  analogy  with  some  other  situation 

the  pilot  htE  experienced  or  heard  shout.  In  the  latter  csss,  note  that  an 
ana Logy  does  not  function  as  a  general  rule' '«■ . g . ,  "In  sll  situations  Like  x 
and  y.  do  ih"but  is  mote  Like  a  hypothesis  that  x  and  y  etc  in  fact  similar: 
"■In  si  tuition  y,  I  did  z  and  it  worked.  If  situation  x  Is  like  situation  y„  z, 
may  work  here  as  well."),  Only  when  prior  knowledge  folly  determines  the 
choice  is  the  mental  model  not  required. 

Secondly,  prior  knowledge  of  a  quite  sophisticated  sort  is  utilized  in  build- 
liqg  the  device  topology,  The  pilot  may  never  have  encountered  this  specific 
Configuration  of  threats,  but  he  may  be  quite  practiced  at  solving  problems  of 
this  kind.  He  thus  knows  what  components  need  to  be  Included  In  the  model 
(e,g.,  awn  aircraft.,  surface-to-air  threats,  terrain)  and  what  parameters  of 
each  will  be  relevant .  he  "learns  how  to  think"  about  such  problems  by  devel¬ 
oping  abstract  schemas  or  scripts  for  building  appropriate  mental  models.. 

Such  abstract  schemas  and  scripts  may  themselves  be  shaped  by  successes  or 
failures  in  the  raaL  environment.  Another  possibility,  which  occurs  both  In 
Science  (Centner  and  Centner,  19-El)  and  in  ordinary  reasoning  (Lakoff  and 
Johnson,  19S0)1  Is  to  construct  mental  models  by  metaphorically  mapping  ob¬ 
jects  and  behaviors  in  one  domain  onto  phenomena  In  another  (e . g . ,  conceiving 
electricity  as  a  fluid,  or  an  argument  as  a  “war"  between  competing 
positions) „ 

A  device  topology  by  itself  is  a  static  structure;  it  must  be  actively  used  If 
the  required  predictions  are  to  be  generated.  The  second  major  concept  in 
deKleer  and  Erown1  s  theory  involves  a  process  called  "■envisioning."  Envision* 
ing  derives  function  from  structure  by  a  process  of  propagation  whereby  one 
starts  with  a  single  input  state  (e.g.,  an  action  option  or  candidate 
exp 1 ana t ion) ,  then  examinee  the  nearby  components  to  observe  its  effects,  ex- 
amiflti  the  nearby  cOwpcnrnts  of  tbdse  components,  and  so  on.  Envisioning 
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results  In  o  dependency  graph  -of  causes  and  effects;  e,g,,a  if  I  da  y 
happens;  as  a  result,  I  do  e,  and  u  happens,  and  so  on,  In  other  words,  en¬ 
visioning  converts  a  representation  in  terms  of  interacting  objects  (the 
device  topology)  into  a  representation  of  a  set  of  temporally  and  causally  re¬ 
lated  event  schemas,  which  deKleer  and  Brown  call  the  "causal  model," 

Although  the  basic  idea  of  envisioning  is  quite  simple,  its  application  may  in 
fact  involve  a  quite  difficult  process  of  problem  solving ,  Thu  difficulty 
arises  because  the  initial  knowledge  oE  device  topology  may  be  iciuf £ i c lent  to 
determine  the  behavi-or  of  the  system  (e.g,„  if  I  do  x,  y  might  happen  but  Z 
night  also  happen)  .  When  this  Is  the  case,  deKleer  and  Brown  propose  that  en¬ 
visioning  el  ruinate  a  the  ambiguity  by  sating  assumptions.  Such  assumptions 
may  concern  the  existence  of  causally  relevant  but  unobserved  attributes ,  the 
temporal  order  of  events,  the  satisfaction  of  rule  conditions,  or  precise  at¬ 
tribute  values ,  Assumptions  may  have  to  be  revised  subsequently  i£  actually 
observed  events  conflict  with  the  events  predicted  by  the  model. 

Once  envisioning  has  produced  s  causal  model  (i,e. ,  a  predicted  sequence  of 
events"),  the  model  can  be  "run"  to  predict  a  specific  event  or  outcome .  Run- 
ning  is  a  relatively  simple  matter  of  activating  &  pre-existing  schema.  The 
main  work  of  problem  solving  at  the  knowledge -baaed  level  has  been  ac¬ 
complished  hy  the  processes  in  which  the  schema  was  created:  i,e.,  construct¬ 
ing  the  device  topology,  generating  an  option,  and  envisioning  its  con¬ 
sequences  . 

2,2.5  Analogical  mode  la  and  uncertainty.  'Je  have  argued  that  in  order  to 
support  the  function  of  gene rating  new  knowledge ,  mental  node Is  must  Involve 
some  Internal  version  of  *  generate -and- test  process;  and  In  order  to  imple¬ 
ment  the  latter,  mental  models,  must  be  composed  of  well-understood  components 
which  are  "glued"  together  in  order  to  observe  their  Interaction.  We  now  con¬ 
sider  an  additional  corollary  of  this  argument,  which  has  implications  both 
for  display  design  and  for  likely  weaknesses  in  natural  human  methods  of 
reasoning.  Mental  models  which  satisfy  the  above  requirements  belong  to  a 
Class  of  modeLs  which  may.  somewhat  loosely,  be  characterized  as  "analogical." 
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There  la  considerable  discussion  and  debate  in  the  research  community  regard¬ 
ing  the  nature  of  {and  the  need  for)  a  distinction  between  " analog"  and 
"propositional"  representations  (e.g,,  Fylyshtn,  1979;  Kosslyn,  i960;  Rumel- 
hart  and  Norman,  1965 ),  Nevertheless,  wt  would  argue  that  a  pi au Bib Is  and  im¬ 
portant  distinction  can  be  made,  based  on  the  requirement  that  mental  models 
have  the  capability  of  generating  new  knowledge. 

Shepard  (1975)  and  Hetzler  and  Shepard  (1974)  have  summarized  empirical 
evidence  concerning  the  properties  of  man  cal  imag.ee  which  appear  to  distin¬ 
guish  them  from  other  internal  represen Cat to ns ,  In  particular,  they  mention: 

o  a  one-to-one  correspondence  between  c opponents  of  the  representation 
and  components  of  the  situation  which  it  represents  (e.g.s  the  image 
of  a  chair  appears  to  have  legs,  a  scat.,  a  back); 

o  a  one- to ’■one  c-0  XT*  spun  dent*  in  time  of  the  states  which  the  repre¬ 
sentation  passes  through  and  the  states  of  the  represented  situation 
(c.g.,  in  imagining  Che  rotation  of  a  three-dimensional  object). 

iJc  would-  argue,  based  on  tur  discussion  in  tht  previous  sections,  chat  bnrJi  □  £ 
chtss  properties  muwC  char acte rite  mental  models F  if  such  models  are  co 
adequately  support  an  internal  generate  and  test  process,  they  must  have  com¬ 
ponents  which  correspond  to  components  in  the  represented  situation,  and 
changes  in  the  model  components  must  causally  mirror  changes  in  the  environ¬ 
ment,  In  both  respects,  such  models  appear  to  differ  from  H pro positional" 
representations,  in  which  (a)  chert  are  syntactic  elements  (like  "the"  and 
"ail")  with  no  direct  representational  function,  and  (b)  changes  in  state  are 
more  typically  represented  by  large,  abrupt  shifts  in  the  representation 
rather  than  a  gradual  transition  through  Intermediate  states, 

Propositional  representations  tan  be  developed  which  mimic  the  behavior  of 
analogical  models,  l.t.,  which  pass  through  an  appropriate  temporal  sequence 
of  intermediate  states,  Rumelhart  and  Norman  (1985)  thus  propose  a  somewhat 
stronger  criterion  for  an  analogical  model  in  addition  CO  che  two  properties 
mentioned  above: 
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o  the  representing  relation  has  the  seme  inherent  f'.&nstraints  as  the 
teptos-cntRij  relation, 

for  example,  suppose  a  pilot  believes  that  ■'an  SA-4  la  score  dangerous  thrm  an 
SA-2"'  Ond  "sn  S h-Z  It  more  dangerous  than  an  SA-7.“  If  he  represents;  these 
beliefs  p rppps 1 C idnal ly  (e.g, ,  in  English  or  In  some  "mental  language"),  ha 
can  inf at  that  "an  SA-4  is  more  dangerous  than  an  SA-7"  only  if  he  also 
believes  some  general  rule  stating  the  transitivity  of  dangerousness  <e.gr3 
“if  A  is  more  dangerous  than  Er  and  E  1e  more  danger ons  than  Ch  then  A  is  more 
dangerous  than  C")u  However,  if  he  represents  these  beliefs  analogically, 

^-E'j  by  placing  tokens  for  5A-£h  5A-4h  and  SA-7  On  a  line  In  positions  which 
represent  their  dangerouetiess  \ 

SA-7  SA-2  5A-4 

— ^  - - - — r 

more  dangerous 

than  the  "inference41  becomes  trivial.  The  relationship  between  SA-4  and  SA-7 
o.™  simply  he  "read  off"  the  model,  once  the  tokens  ate  placed  appropriately 
to  represent  his  initial  beliefs,  The  reason,,  of  course,.  is  that  heing-to- 
the -right- of  and  being- mo  re -dangerous  -  than  have  the.  same  inherent  c  nns  cfs  ints 
trans it ivlty ) , 

We  would  argue  that  the  additional  criterion  proposed  by  Humelhart  and  No man 
must  also  be  satisfied  by  mental  models,  if  they  ate  to  have  the  capacity  to 
generate  new  knowledge.  Tills  is  simply  the  requirement ,  discussed  in  the  last 
section,  that  there  be  no  pre-existing  rule  describing  the  interaction  of  the 
components,  "Inherent"  constraints  in  the  deKleer  and  Brown  framework  do  not 
arise  siiaply  from  the  representational  format  (e.gr,  A  line),  but  are  due  to 
properties  of  thu  "conduits"  that  connect  objects  in  the  model, 

Johnson- Laird  {1983)  has  recently  defined  a  concept  of  "mental  model"  directly 
in  terms  of  these  analogical  properties,  In  particular,  according  to  Johnson - 
Lsird,  what  distinguishes  a  mental  model  from  other  forms  of  knowledge  repre¬ 
sentation  is  the  close  structural  isomorphism  between  the  codel  and  the  state 
of  affairs  it  represents .  Every  element  in  the  uental  model  plays  a  symbolic 
(rather  than  a  Purely  formal)  role,  Kor  example,  in  a  semantic  network 
numerous  formal  devices  are  required  1,0  represent  a  simple  generalisation  like 


11  Every  aircraft  of  type  y  ha$  ECM.  gear'  (a,g, ,  abstract  nodes  eo rra ep and in^  to 
the  set  of  *11  x-cype  aircraft,,  the  set  of  all  ECM  gear,  and  the  set  of  all 
"having 11  or  "containing"  relations;  partitions  of  the  network  into  components 
corresponding  to  the  antecedent  and  consequent  of  the  proposition,  etc. ) ,  A 
mental  model  of  the  same  fact,  by  contrast,  might  involve  tokens  symbolic ing 
M-type  aircraft  and  tokens  symbolizing  ECU  gear  associated  with  one  another  by 
symbols  representing  containment; 

aircraft  ■*  ECM  gear 
x- aircraft  ■+  Edi  gear 
X- aircraft  w  ECH  gaar 
(ECM  gasr) 

Parentheses  are  placed  around  one  of  the  ECM.  gear  tokens  to  represent  thfe  fate 
that  some  ECM  geer  may  be  present  on  other  types  of  aircraft.  The  key  feature 
of  a  mental  model,  according,  to  Johnson -Laird,  is  the  economy  and  naturalness 
of  the  representation  it  iopeses. 

fJhch  new  information  is  obtained,,  it  is  not  simply  appended  to  a  list  of 
beliefs i  iv  is  added  directly  to  the  appropriate  mental  model.  Fotf  example, 
on  learning  that  '"There  is  ah  X-typ*  *£rct*fft  it  y  t Laid."  ,  u*  jet; 

y 'field  aircraft  -«  x-aireraf c  ■+  ECM.  .gear 

K-aircrafc  -+  ECM  geac 
x-airersft  ECM  gear 
(ECK  geap) 

The  ** inf e rente71  that  there  Is  on  aircraft  with  ECM  gear  at  y  field  can  now  he 
directly  tend  off  the  updated  model.  Thus,  Johnson -Laird1  s  primary  inrerest 
to  mental  models  is  to  explain  features  of  human  cognition  that  $c=m  incom¬ 
patible  with  an  account  of  problem- solving  strictly  in  terms  of  abstract 
reasoning,  or  application  of  general  rules. 

Analogical  models  are  not  necessarily  constrained  in  Che  kinds  of  thing*  they 
can  represent,  only  in  the  vay  those  things  ate  represented.  Thus,  Johnson - 
Laird  distinguishes  between  physical  models  which  rep resent  perceived  objects 


21* 


and  relations ,  and  con c ep £ua J  models  which  represent  non- perceptual  relation¬ 
ships.  It  seems  clear  that  pilot  displays  in  future  aircraft  systems  will  in¬ 
volve  both  types  of  models.  Pilot  functions  (flying  the  Hitcraft.  operating 
sensors,  planning  and  executing  tactics)  all  require  the  formation  of  mental 
models  of  the  aircraft  in  relation  to  the  physical  world  of  targets,  sensors, 
weapons  and  environment,  Equally  important,  however,  ate  conceptual  models, 
which  represent  non -perceived  relationships  such  £6  "able  to  detect,"  "able  to 
jam,11  or  "able  to  hit," 

J obptuiv Lit rd  identifies  six  types  of  physical  models;  relational  models  (a 
static  frame  containing  a  finite  set  of  entities,  properties,  and  relations); 
spatial  models  (in  which  all  relations,  both  represented  and  representing,  are 
spatial);  temporal  models  (representing  a  sequence  of  events  ox  spatial  situa¬ 
tions  in  time),  kinematic  models  (a  temporal  model  that  is  psychologically 
Continuous) ,  dynamic  models  (a  kinematic  model  that  incorporates  causal 
relations)  and  images  (a  viewer -centered  representation  of  an  underlying 
three-dimensional  spatial  or  kinematic  model). 

In  addition,  Johns  on -Laird  (1-983)  describe*  four  type*  of  conceptual  model; 

1.  H&nadlc ,  representing  asaer Cions  about  individual  entities,  their 
properties,  and  identities  between  them; 

2,  Relational,  which  introduce  a  finite  number  of  relations  between  the  en^ 
titles  £n  a  monadic  code 1  (such  as  "there  are  more  s's  than  b's"); 

3-.  He ta- linguistic ,  which  introduce  semantic  relationships  such  as  “refers 
to,"  “means, "  "is  called,  11  etc,; 

4.  Set  “-theoretic  ,  which  includes  notions  of  set  -  rtembersh  ip  ,  sot  properties, 
and  relations  among  sets. 

In  terms  of  this  taxonomy,  deKleer  and  Brown's  "device  topology"  i£  a  Special 
kind  of  cone eptual/rel ational  model,  In  which  tokens  are  related  to  one 
another  by  potential  causal  effects,.  Envisioning  then  derives  a,  temporally 
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and  causally  related  sequence  of  events;  i„e,,  e  ■causal  model11  (for  ceKLeer 
and  Brown)  -  a  phy s  1  cs  1/dynan.i c  model  (for  Johnson -laird)  , 

Un  certs  In  ty .  Despite  this  flexibility  In  the  type*  of  objects  And  relation’ 
ship  that  oan  bo  represented,  the  constraints  Imposed  by  the  nature  of 
analogical  models  have  important  consequence*,  Most  important,  we  think,  is 
the  difficulty  that  is  implied  in  the  representation  of  indeterminancy , 
whether  uncertainty  about  facts  or  about  values.  Suppose,  for  example, ,  that 
we  know  that  "Base  A  is  west  of  flase  C"  end  " Base  A  is  west  of  Base  ft . "  How 
can  we  represent  this  in,  an  analogical  model?  He  have  two  choices; 

ABC 

or 

ACE. 

Our  information,  does  not  specify  the  relationship  between  B  and  G.  However, 
the  strict  requ.irec.ent  of  isomorphism  in  the  analogical  model  forces  us  to 
choose.  He  cannot  have  a  model  with  a  direct  mapping  to  the  state  of  affairs 
it  represents  when  we  do  not  know  what  that  state  o£  affairs  is. 

A  similar  difficulty  arises  in  the  representation  of  uncertainty  about  values. 
Suppose,  for  example ,  that  a  pilot  is  considering  three  tactical,  options.  In 
terms  o£  risk  to  own.  aircraft,  option  C  is  better  than  option  B  which  is  hot¬ 
ter  than  option  A,  But  in  terms  of  time  and  fuel  required  to  execute  the  tac¬ 
tic,  E  i*  better  than  C  which  is  better  than  L  The  pilot  can  conclude  chat  C 
is  better  than  h  (since  it  Is  superior  both  in  ten*®  of  risk  and  in  teras  of 
time  and  fuel) ,  and  that  3  1*  better  than  A  (since  it  too  is  superior  on  both 
dimensions) ,  But  he  does  not  know  whether  ft  is  preferable  to  G  or  C  is 
preferable  to  ft, 

The  strict  requirentent  of  isomorphism  can  be  relaxed  in  various  ways  to  repre¬ 
sent  indeterminacy  (cither  about  facts  or  ;ibout  values),  but  each  approach 
has  its  drawbacks; 

o  Use  nf  aui  ti  pin  models -  - e , g .  , 

'A  ft  c' 

A  C  ft. 


£6 


The  problem  here  Is  the  potential  combinatorial  explosion  as  r)$v  In¬ 
formation,  and  new  indeterminacies ,  are  added. 


where  the  arrows  represent  "to  the  west  of**  or  “is  worse  than,*  The 
problem  here  is  that  the  naturalness  of  the  mental  model  approach  is 
losCJ  LTif-erenn^s  cnn  no  longer  be  directly  read  off  the  modal h  since 
the  spatial  relations  In  the  model  -are  no  longer  being  used  rap-re^ 
sent  at  ior.alTy . 

o  Utilisation  of  more  imprecise  mo-dels  - -e.  g .  „ 

A  [EG] 

Eere  LsomorphisiL  is  preserved,  but  E  and  C  are  lumped  together  as  a 
single  token.  This  may  be  a  viable  approach,  unless  decision  making 
requires  that  the  relative  locations  or  values  of  E  and  C  be  known, 

o  Adoption  of  one  model  by  35 sumption ,  with  subsequent  revision  if 
necessary- -e.g,  p 

Assume ,  a  e  £ 


This  is  perhaps  the  most  common  metliod,  The  danger,  of  course,  is 
that  we  u^ay  lose  track  of  (or  be  unaware  of  J  our  assumptions  and 
feel  an  unwarranted  sense  of  certainty, 

Ey  contrast  with  analogical  medals,  normative  approaches  represent  uncertainty 
by  ciathema t itally  aggregating  the  possibilities ,  thus  providing  an  abstract 
level  of  representation  chat  corresponds  to  no  actually  realisable  state  of 
affairs.  For  decision  asking  in  the  context  of  uncertainty  about  facts,  an 
"expected  value11  is  computed  for  each  option:  i.e.*  a  weighted  average  of  the 
possibilities ,  in  which  the  probabilities  assigned  to  each  possible  outcome 
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serve  as  the  weights  (cf . ,  Raiffa,  1968).  For  uncertainty  about  values,  a 
"multiattribute  utility"  score  is  computed;  i.e.,  a  weighted  average  of  the 
scores  on  different  evaluative  dimensions,  in  which  measures  of  the  relative 
importance  of  differences  in  each  dimension  serve  as  the  weights  (cf . ,  Keeney 
and  Raiffa,  1976).  Abstractions  such  as  these  can  play  no  role  in  a  pilot's 
mental  models  of  the  world  since  they  are  averages  concocted  for  a  particular 
occasion,  not  real  or  even  possible  events;  hence,  despite  their  value  in 
decision  making,  they  cannot  be  utilized  effectively  to  increase  causal  under¬ 
standing  of  the  situation  (i.e.,  what  will  happen  and  when). 

The  main  weakness  of  mental  models  (their  failure  to  represent  uncertainty)  is 
thus  a  by-product  of  their  defining  characteristics  (the  direct  or  analogical 
representation  of  states  of  affairs)  and  is  for  that  reason  intimately  as¬ 
sociated  with  their  strength  (the  ability  to  generate  new  knowledge) .  In 
Section  2 . 3  we  will  turn  to  some  implications  of  this  "weakness"  for  the 
manipulation  of  uncertainty  in  unaided  problem  solving. 

2.2.4  Hierarchical  knowledge  and  the  nature  of  expertise.  A  major  variable 
in  the  performance  of  a  combat  aircraft  is  the  level  of  knowledge  and  ex¬ 
perience  of  the  pilot.  It  is  reasonable  to  suppose,  therefore,  that  pilot 
displays  for  intelligent  avionics  should  take  into  account  and  be  tailored 
toward  the  level  of  expertise  of  a  particular  user  (cf . ,  Cohen  et  al,  1982; 
Cohen  et  al.,  1985).  In  this  section,  we  briefly  consider  how  the  knowledge 
structures  considered  above  might  differ  between  novices  and  experts.  It 
turns  out  that  the  notion  of  hierarchical  organization  plays  a  key  role. 

Figure  2-3  (which  may  be  compared  with  Figure  2-1)  summarizes  the  implications 
of  our  discussion  for  Rasmussen's  basic  framework  of  cognitive  performance. 
"Rule-based"  performance  had  been  replaced  by  a  concept  of  stereotypical  per¬ 
formance,  which  Incorporates  hierarchical  structure  and  emphasizes  top-down 
processes  by  means  of  which  higher  level  goals  and  schemas  may  activate  lower 
level  sub -goals  and  schemas.  Thus,  as  pilots  accumulate  experience,  they  may 
acquire  more  elaborate  "is -a"  and  "is-a-part-of"  knowledge  structures.  More 
extensive  higher-level  knowledge  and  top-down  processes  (i.e.,  scripts)  will 
permit  them  to  interpret  situations  more  rapidly,  to  anticipate  events,  and 
adopt  longer  time  horizons  of  planning.  At  the  same  time,  more  extensive 
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Figure  2-3:  Modified  Version  of  Rasmussen's  Framework 


lower  level,  bottom-up  knowledge  will  permit  finer  discriminations  among 
situations  and  more  appropriate  responses. 

Stereotypical  knowledge  embodied  in  scripts  does  not  represent  the  causal 
relationships  underlying  associations  between  goals  and  subgoals.  Mental 
models  must  therefore  be  called  upon  to  discover  new  ways  of  achieving  goals 
when  existing  knowledge,  at  any  level,  proves  inadequate  (Leddo,  et  al., 

1987) .  It  follows  that  mental  models  themselves  may  differ  hierarchically 
both  on  the  "is-a"  and  "is -a-part-of "  dimensions;  i.e.,  in  terms  of  the  scope 
and  generality  of  the  objects  which  they  causally  relate.  A  pilot,  for  ex¬ 
ample,  may  take  evasive  action  to  avoid  threats  enroute  to  the  target  and 
thereby  jeopardize  his  chances  of  arriving  at  the  target  by  the  designated 
time.  The  pilot  may  then  use  relatively  low  level  mental  models  to  explore 
alternative  shorter  routes  or  alternative  faster  speeds,  running  such  models 
to  determine  if  various  options  achieve  the  appropriate  time  over  target  while 
at  the  same  time  incurring  acceptable  risk.  If  it  appears,  however,  that  ar¬ 
riving  at  the  target  by  the  designated  time  with  acceptable  risk  is  not  pos¬ 
sible,  higher  level  mental  models  may  be  utilized  to  decide  whether  ultimate 
mission  objectives  (damage  the  enemy,  return  safely)  can  best  be  achieved  by 
continuing  to  the  original  target,  aborting  the  mission,  or  seeking  a  secon¬ 
dary  target  instead.  More  experienced  pilots  would  be  expected  to  have 
developed  schemas  and  procedures  which  facilitate  the  construction  of  such 
models,  and  which  facilitate  the  selection  of  plausible  options  for  testing. 

A  third  sense  in  which  pilot  knowledge  is  hierarchical  is  represented  by  the 
classification  of  performance  levels  itself;  i.e.,  skill-based,  stereotypical, 
and  knowledge -based.  We  observed  above  (Section  2.1)  that  increasing  exper¬ 
tise  is  often  characterized  by  a  shifting  of  levels  from  knowledge -based  to 
rule-based  to  skill-based.  It  is  worth  noting  that  the  boundary  between 
stereotypical  performance  and  knowledge-based  performance  is  not  altogether 
clearcut.  In  highly  unfamiliar  situations,  knowledge -based  behavior  involves 
the  construction  of  a  new  device  topology;  i.e.,  basic  components,  their 
states,  and  their  interconnections.  However,  it  may  be  possible  to  deal  with 
less  novel  situations  by  utilizing  a  pre-existing  model  and  revising  some  of 
the  assumptions  made  during  "envisioning,"  i.e.,  a  new  temporal/causal 
sequence  of  events  may  need  to  be  derived  from  the  existing  device  topology. 
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In  still  more  familiar  contexts,  it  may  not  be  necessary  to  re -envision  the 
causal  model;  it  may  only  be  necessary  to  "run"  an  existing  causal  model,  with 
new  inputs  representing  the  changed  circumstances  or  options.  Finally,  in  the 
extreme  case  of  stereotypical  performance,  the  stored  results  of  previous  run¬ 
nings  of  the  causal  model  may  be  retrieved  to  solve  the  present  problem. 

Ironically,  although  experienced  pilots  should  be  more  skilled  at  building 
mental  models,  they  should  at  the  same  time  have  less  need  to  do  so.  As  the 
number  of  situations  with  which  they  are  familiar  increases,  experienced 
pilots  have  less  need  to  call  upon  deeper  causal  analysis.  Larkin  et  al . 

(1980)  found  that  in  solving  physics  problems,  for  example,  sophisticated 
novices  worked  backward  from  the  unknown  through  various  subgoals  to  the  given 
quantities  and  explicitly  mentioned  the  equations  used  at  each  stage.  Ex¬ 
perts,  by  contrast,  were  faster,  worked  forward  from  the  given  to  the  desired 
quantities,  and  usually  verbalized  only  numerical  results  rather  than  the 
equations  used  to  derive  them.  These  results  suggest  that  sophisticated 
novices  can  apply  generate -and- test  methods  (to  discover  ways  of  reducing  the 
gap  between  a  goal  and  a  subgoal) ,  but  that  experts  have  already  embodied  the 
results  of  such  knowledge -based  reasoning  within  stereotypical  procedures. 
Other  evidence  supports  the  idea  that  expert  stereotypical  representations 
reflect  the  properties  of  the  mental  models  from  which  they  were  derived.  Chi 
et  al.  (1981)  found  that  physics  experts  and  novices  differ  in  the  way  they 
sort  problems  by  similarity.  Novices  categorize  problems  by  "surface  struc¬ 
ture,"  i.e.,  superficial  features  such  as  type  of  apparatus,  while  experts 
rely  on  basic  principles  of  physics  and  generic  solution  techniques  associated 
with  such  principles.  Similarly,  algebra  experts  sort  problems  by  solution 
method,  while  novices  depend  on  words  or  objects  mentioned  in  the  problem 
statement  (Schoenfeld  and  Herrmann  (1982) . 

In  sum,  there  are  a  variety  of  avenues  by  which  extended  experience  might  af¬ 
fect  and  improve  pilot  performance  within  the  framework  we  have  described: 

1.  The  direct  accumulation  of  stereotypical  knowledge  in  situations 

which  permit  (a)  generalizing  and  aggregating  lower  level  knowledge, 
and  (b)  refining  and  discriminating  higher  level  knowledge; 


31 


2.  The  development  of  stereotypical  knowledge  about  how  to  solve  novel 
problems,  i.e.,  increasing  the  ability  to  build  and  use  mental 
models;  and 

3.  The  derivation  of  new  stereotypical  knowledge  by  building  and  run¬ 
ning  causal  mental  models  and  storing  the  results . 

2.2.5  Behavioral  decision  theory.  In  this  section  we  turn  to  a  second  body 
of  research,  primarily  concerned  with  human  processes  of  inference  and  choice. 
This  work  has  by  and  large  focused  on  errors  and  biases  in  those  processes , 
and  has  been  less  concerned  to  develop  explanatory  models  of  why  such  errors 
occur.  It  is  well  beyond  the  scope  of  this  report  to  attempt  to  provide  such 
an  explanation.  Our  objective,  rather,  is  to  suggest  that  the  findings 
described  above  on  mental  models  in  knowledge -based  and  stereotypical  reason¬ 
ing  can  illuminate  the  nature  of  the  errors  that  are  observed  and  may  provide 
the  seeds  of  an  eventual  explanation. 

Figure  2-4  provides  a  convenient  framework  for  organizing  the  discussion  of 
errors  in  reasoning.  It  conceptualizes  the  decision-making  process  quite 
generally  as  consisting  of  a  specific  set  of  cognitive  tasks.  First,  goals  or 
objectives  must  be  known  or  identified.  Secondly,  current  circumstances  in¬ 
sofar  as  they  are  relevant  to  the  achievement  of  goals  are  assessed.  If  a 
discrepancy  is  perceived  between  goals  and  reality,  options  for  action  may  be 
generated.  If  more  than  one  option  is  available  a  choice  must  be  made. 

This  is  by  no  means  a  rigid  sequence:  the  process  can  be  iterative  (for  ex¬ 
ample,  revising  goals,  reassessing  the  situation,  or  generating  new  options 
when  the  choice  process  fails  to  turn  up  an  acceptable  alternative);  and  steps 
may  be  skipped  (in  particular,  in  stereotypical  behavior  when,  for  example, 
the  appropriate  action  is  known  based  on  past  experience  with  similar 
situations).  Nevertheless,  this  framework  covers  the  basic  set  of  pos¬ 
sibilities  in  a  decision  situation  and,  moreover,  identifies  the  specific 
aspects  of  human  performance  where  decision- aiding  may  be  of  use. 

It  is  convenient  to  break  each  of  these  major  tasks  down  into  more  specialized 
cognitive  subtasks.  For  example,  situation  assessment  consists  of  collecting 
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Figure  2-4:  Potential  Cognitive  Subtasks  in  the  Decision-Making  Process. 


and  viewing  data  or  evidence,  deriving  inferences,  developing  some  sense  of 
confidence  in  the  conclusions,  and  continuing,  perhaps,  to  draw  further 
higher-level  inferences.  Again,  the  steps  may  be  iterative,  may  be  combined, 
or  may  be  skipped  altogether  by  some  decision  makers  in  some  situations. 

(Note  that  the  term  "evidence"  is  quite  relative;  evidence  in  one  process  may 
be  the  highly  uncertain  conclusion  of  a  prior  analysis.)  This  decomposition 
of  cognitive  sub tasks  could,  of  course,  be  continued.  It  has  been  postulated 
that  all  cognitive  functioning  can  ultimately  be  analyzed  into  a  set  of  simple 
"elementary  information  processes"  (Newell  and  Simon,  1972;  Chase,  1978)  such 
as  selecting  an  input,  reading  the  value  of  a  variable,  comparing  two  values, 
and  eliminating  an  alternative. 

Each  of  the  cognitive  subtasks  identified  in  Figure  2-4  has  been  associated, 
at  least  in  laboratory  research,  with  characteristic  shortcomings  in  reason¬ 
ing.  The  following  outline  is  highly  incomplete  and  is  only  meant  to  touch  on 
some  of  the  issues  that  bear  directly  on  the  present  work.  Three  important 
themes,  however,  should  emerge:  (1)  Unaided  decision  processes  employ 
simplifying  heuristics  that  at  best  only  approximate  prescriptively  accepted 
rules  (e.g.,  Bayesian  probability  theory,  multiattribute  utility  theory);  (2) 
a  typical  effect  of  such  heuristics  is  that  awareness  of  uncertainty  about 
facts  or  about  values  is  suppressed;  and  (3)  in  many  instances,  biases  are  a 
result  of  (otherwise  successful)  efforts  to  utilize  natural  knowledge  struc¬ 
tures  and  processes  of  reasoning. 

(a)  Collecting  information.  Wason  (1960),  Einhorn  (1980),  and  others  have 
shown  that  people  tend  to  stubbornly  hold  to  a  hypothesis  generated  early, 
avoid  stringent  tests  of  the  favored  hypothesis,  and,  in  fact,  seek  confirming 
evidence.  People  also  fail  to  collect  evidence  regarding  alternative  causes 
of  an  event,  where  more  than  one  cause  is  possible  (Shaklee  and  Fischhoff, 
1982).  These  findings  may  reflect  the  utilization  of  analogical  models,  which 
are  isomorphic  with  the  states  of  affairs  they  represent  and  therefore  fail  to 
provide  an  effect  representation  of  indeterminancy.  They  may  also  reflect  the 
burden  on  short  term  memory  and/or  processing  capacity  of  generating  and 
manipulating  more  than  one  causal  mental  model. 
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(b)  Inferring  conclusions .  A  number  of  studies  show  that  a  statistical  model 
of  a  person's  judgment  process  can  outperform  (in  accuracy)  that  person's  own 
judgments,  thus  suggesting  that  people  do  not  effectively  utilize  the  informa¬ 
tion  available  to  them  in  inference  tasks  (Dawes,  1975;  Cohen,  1982).  People 
tend  to  ignore  later  evidence  that  contradicts  a  favored,  or  earlier,  datum 
and  to  double  count  redundant  evidence  (Schum  and  Martin,  1981)  .  People  com¬ 
monly  ignore  statistical,  or  "base  rate,"  data  and  overweight  unique  or 
problem- specific  evidence,  which  is  more  readily  subject  to  causal  modeling 
(Kahneman  and  Tversky,  1972) .  The  significance  of  exceptions  in  a  series  of 
observations  is  often  exaggerated,  i.e.,  treated  as  causally  relevant  rather 
than  the  result  of  sampling  error,  and,  as  a  result,  significant  conclusions 
are  overlooked  (Tversky  and  Kahneman,  1971).  These  observations  suggest  the 
predominance  in  natural  reasoning  of  non- statistical ,  causal  mental  models 
(Johnson,  1985).  When  people  do  attempt  to  make  statistical  judgments, 
moreover,  estimates  may  be  biased  by  the  ease  of  recall  (or  "availability")  of 
instances  of  a  particular  class  of  events  in  a  mental  sampling  (Tversky  and 
Kahneman ,  1972). 

(c)  Assessing  quality  of  conclusions .  A  number  of  studies  show  that  people 
consistently  overestimate  their  degree  of  certainty  regarding  predicted  events 
and  estimated  quantities ,  even  in  areas  where  they  are  (rightfully)  regarded 
as  experts  (Kadane  and  Lichtenstein,  1982) .  When  inference  proceeds  in  stages 
(e.g.,  deriving  the  probability  of  being  detected  by  a  ground  radar  site  from 
information  about  its  classification  and  range) ,  people  often  simplify  the 
process  by  acting  as  if  conclusions  at  earlier  stages  (classification  and 
range)  were  known  to  be  true,  rather  than  merely  inferred  (Schum,  et  al. , 

1973) .  These  results  also  seem  to  reflect  the  difficulty  of  representing  am¬ 
biguous  states  of  affairs  in  analogical  models.  Similarly,  the  probability  of 
a  detailed  hypothesis  or  scenario  is  likely  to  be  judged  higher  than  the  prob¬ 
abilities  for  its  components  (Tversky  and  Kahneman,  1983) .  The  latter  effect 
may  arise  because  additional  details  increase  the  match  between  the  hypothesis 
and  user  mental  models  (Leddo  et  al.,  1984). 

(d)  Generating  options.  Ingrained  ways  of  viewing  a  problem  (e.g.,  pre¬ 
existing  schemas  or  mental  models)  tend  to  hinder  the  generation  of  novel  and 
creative  solutions.  Gettys  and  Fisher  (1979)  and  Gettys  et  al.  (1981)  have 
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shown  that  people  often  overlook  important  subsets  of  the  available  options  or 
hypotheses.  Moreover,  people  segment  complex  options  into  "natural"  com¬ 
ponents  (possibly  based  on  distinct  causal  relationships) ,  and  treat  the  ele¬ 
ments  as  if  they  were  independent  choices,  leading  to  suboptimal  choices 
(Tversky  and  Kahneman,  1981).  There  is  a  tendency  to  formulate  options  within 
a  short  time  frame  and,  as  a  result,  to  overlook  the  cumulative  risk  of  pursu¬ 
ing  a  course  of  action  over  a  long  period  of  time  (Slovic,  et  al.,  1978). 

This  may  reflect  the  difficulty  of  "running"  mental  models  to  simulate  events 
far  in  the  future,  and  the  (related)  absence  of  high-level  aggregated  schemas 
for  novel  activities  of  long  duration.  Individuals  differ  in  the  degree  to 
which  they  consider  future  choices  in  current  planning  (Streufert  and 
Streufert,  1981)  and  in  the  number  of  options  they  generate  (Driver  and  Mock, 
1976) . 

(e)  Assessing  uncertainty  of  outcomes.  When  predictions  are  made  about  the 
outcome  of  an  option,  there  may  be  effects  of  "wishful  thinking"  (e.g.,  higher 
probability  assessments  for  high  utility  outcomes)  or  overcautiousness  (e.g., 
lower  assessments  for  high  utility  outcomes)  (Einhorn  and  Hogarth,  1984).  The 
size  of  these  effects  may  depend  on  the  perceived  uncertainty  of  the  predict¬ 
ion,  and  may  reflect  a  process  of  making  assumptions  to  reduce  indeterminancy. 
Perceived  uncertainty  in  turn  might  depend  on  the  degree  to  which  available 
evidence  matches  user  schemas.  The  "gambler's  fallacy,"  involving  distorted 
conceptions  of  randomness,  may  be  a  by-product  of  powerful  top-down  or 
expectancy -driven  processes  of  pattern  recognition  (Lopes,  1982). 

(f )  Assessing  value  of  outcomes.  Decision  makers  do  not  typically  consider 
all  the  potential  outcomes  of  an  action  together.  Rather,  outcomes  are 
grouped  into  "mental  accounts"  corresponding  to  natural  objects  or  causal 
relations,  and  choices' may  depend  critically  on  the  particular  grouping  that 
is  adopted  (Kahneman  and  Tversky,  1982).  Perhaps  the  best  known  research  on 
choice  behavior^ under  risky  conditions  is  that  of  Tversky  and  Kahneman  (1981) 
who  have  shown  that  decisions  are  significantly  influenced  by  the  way  the 
problem  is  framed.  A  key  feature  of  this  work  is  that  people  naturally  repre¬ 
sent  outcomes  in  causally  relevant  terms,  by  the  difference  it  would  make 
relative  to  some  reference  point.  Formally  equivalent  choice  problems  will  be 
responded  to  differently  depending  on  whether  the  outcomes  are  presented  as 
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gains  (e.g.,  lives  saved  relative  to  a  worst  case  reference  point)  or  losses 
(e.g.,  lives  lost  relative  to  the  status  quo).  People  tend  to  be  risk-averse 
for  gains  and  risk- seeking  for  losses,  so  that  problem  framing  can  have  an  im¬ 
portant  impact  on  choice  behavior. 

(g)  Selecting  an  option.  Choice  heuristics  may  be  adopted  which  reduce  the 
amount  of  information  which  decision  makers  utilize.  In  Elimination  by 
Aspects  (Tversky,  1972),  for  example,  attributes  are  considered  serially  in 
order  of  importance;  options  falling  below  a  cut-point  on  an  attribute  are 
eliminated  at  each  stage,  and  not  considered  further;  and  the  process  stops 
when  the  set  of  options  has  been  reduced  to  the  desired  number.  This  strategy 
is  consistent  with  the  use  of  causal  mental  models  to  predict  the  achievement 
or  non- achievement  of  a  goal  on  each  attribute.  In  particular,  if  each  at¬ 
tribute  were  associated  with  a  different  mental  model  (for  example,  time  to 
get  to  the  target  might  be  predicted  in  one  model,  risk  to  the  aircraft  from 
ground  threats  in  another) ,  then  organizing  information  processing  in  this  way 
minimizes  the  need  to  switch  back  and  forth  between  models.  The  problem,  of 
course,  is  that  tradeoffs  between  goals  are  not  considered;  in  this  strategy, 
an  option  might  be  eliminated  for  missing  a  cut-point  on  one  dimension  even 
though  it  scores  very  high  on  other  dimensions.  Research  by  Lopes  (1986)  sug¬ 
gests  that  some  decision  makers  compare  options  only  in  terms  of  their  perfor¬ 
mance  in  the  "worst  case"  outcome  and  disregard  performance  on  other  dimen¬ 
sions,  e.g.,  non-worst  case  outcomes. 

An  important  theme  in  many  of  these  findings  is  that  biases  are  a  result  of 
people's  efforts  to  utilize  natural  knowledge  structures  and  processes  of 
reasoning.  More  specifically,  a  persuasive  case  can  be  made  that  biases  arise 
from  the  properties  of  mental  models;  (a)  the  requirement  of  a  one-to-one 
mapping  between  elements  of  the  model  and  elements  of  the  situation  which  they 
represent;  (b)  facilitation  of  the  ability  to  "run"  a  single  mental  model,  at 
the  expense  of  the  ability  to  manipulate  multiple  mental  models 
simultaneously;  and  (c)  the  substitution  of  "inherent"  relations  for  general 
rules  of  inference  (such  as  in  Bayesian  probability  theory).  Ue  have  argued 
that  all  of  these  properties  are  essential  for  the  function  of  mental  models 
in  generating  genuinely  new  knowledge. 
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A  common  rationale  for  including  humans  in  command  and  control  systems  is  that 
they  are  more  "flexible"  than  machines.  This  is  presumed  to  mean  that  humans 
can  quickly  perceive  new  patterns  or  trends  in  a  situation  as  it  develops,  and 
generate  new  hypotheses  and  new  options  that  are  responsive  to  the  new 
conditions.  As  we  have  seen,  there  are  important  limitations  to  this  conclu¬ 
sion.  Nevertheless,  in  high-level  problem  solving,  methods  for  tapping  a 
user's  knowledge  will  often  be  an  important  element  in  the  success  of  a  com¬ 
puterized  system.  For  example,  current  (and  foreseeable)  artificial  intel¬ 
ligence  technology  falls  short  of  human  capabilities  in  reasoning  on  multiple 
levels,  solving  novel  problems  (Newell,  1981),  handling  unanticipated  types  of 
evidence,  and  using  concepts  like  causality,  intention,  and  belief  (i.e.,  men¬ 
tal  models  of  other  agents)  (Buchanan,  1981;  McCarthy,  1977).  The  results 
summarized  above  imply  that  techniques  for  exploiting  such  knowledge  must 
guard  against  serious  potential  pitfalls.  Thus,  the  design  of  interactive 
decision- aiding  functions  demands  a  precarious  balancing  act  between  encourag¬ 
ing,  on  the  one  hand,  and  modifying,  on  the  other,  a  user's  natural  procedures 
for  handling  information. 

2 . 3  Personalized  and  Prescriptive  Decision  Support:  A  Generalized  Display 
Design  Concept 

In  this  section  we  draw  together  the  threads  of  the  previous  discussions,  and 
present  a  design  concept  for  interactive  displays  which  is  based  on  insights 
both  from  the  literature  on  knowledge  representation  and  the  literature  on  be¬ 
havioral  decision  theory.  This  design  concept,  referred  to  as  Personalized 
and  Prescriptive  Decision  Support  (PDS) ,  permits  adaptation  of  a  system  to 
both  the  decision  maker  (to  achieve  cognitively  compatible  displays)  and  the 
decision  situation  (to  avoid  biases),  and  utilizes  both  automatic  system  pro¬ 
cedures  and  user  choice  in  making  the  adaptation.  The  present  discussion  is 
based  on  Lehner,  et  al  (1987);  earlier  descriptions  are  contained  in  Cohen  et 
al.,  1982;  Cohen  et  al. ,  1985;  and  Cohen  et  al.,  1986a). 

This  approach  is  in  part  a  response  to  the  behavioral  decision  making  litera¬ 
ture  (discussed  in  Section  2.3)  that  suggests  that  human  judgment  and 
decision-making  behavior  are  subject  to  a  number  of  cognitive  biases.  For  in¬ 
stance,  we  saw  that  in  making  choices,  people  often  set  cutoffs  on  separate 
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dimensions  (and  fail  to  consider  tradeoffs),  consider  only  some  of  the  pos¬ 
sible  outcomes  of  an  option,  etc.  The  existence  of  cognitive  biases  is  often 
used  as  an  argument  in  favor  of  the  need  for  decision  aids.  The  typical  ap¬ 
proach  to  aiding,  however,  is  to  supplant  the  user's  unaided  method  for 
solving  the  problem  with  a  normative  method,  and  to  replace  human  judgment 
regarding  the  solution  with  the  judgments  provided  by  a  normative  model  em¬ 
bedded  within  the  aid. 

By  contrast,  a  major  premise  of  Personalized  and  Prescriptive  Decision  Support 
is  that  user-preferred  methods  may  have  significant  utility  along  with  their 
flaws .  Users  may  employ  internal  models  that  embed  valuable  knowledge  of  the 
problem  domain  accumulated  over  many  episodes  of  experience.  User  mental 
models  which  are  ideally  tuned  to  capture  complex  causal  relationships  may, 
however,  be  quite  poor  at  representing  uncertainty  or  balancing  tradeoffs  be¬ 
tween  competing  goals.  Thus,  cognitive  biases  may,  in  some  cases,  represent 
the  downside  of  powerful  human  information-processing  capabilities.  Tradi¬ 
tional  decision  aiding  may  "throw  out  the  baby  with  the  bath  water"  in  forcing 
users  to  avoid  biases  by  adopting  unfamiliar  modes  of  reasoning  and  repre¬ 
senting  information.  The  aim  of  Personalized  and  Prescriptive  Decision  Sup¬ 
port  is  to  substitute  a  more  precise,  "surgical"  removal  of  biases- -by  reduc¬ 
ing  biases  in  the  context  of  the  decision  maker's  preferred  approach  to  the 
problem.  The  goal  of  decision  aiding  is  to  retain  the  advantages  of  the  user- 
preferred  method  (i.e.,  more  effective  exploitation  of  user  knowledge)  while 
producing  bottom-line  performance  that  satisfies  normative  constraints. 

The  PDS  approach  to  the  design  of  decision  aids  varies  in  form  depending  on 
whether  adaptation  to  the  decision  maker  or  to  the  situation  is  primary.  (1) 
In  the  former  case,  the  primary  source  of  initiative  is  the  user,  who  deter¬ 
mines  what  basic  modes  of  representing  and  processing  information  will  be 
used.  The  aid,  however,  provides  a  prescriptive  back-up  for  this  user- 
initiated  personalization.  One  form  of  back-up  involves  monitoring  the  user's 
performance,  comparing  it  to  an  internal  normative  model,  and  providing 
prompts  when  the  user-selected  strategy  is  likely  to  lead  to  seriously  subop- 
timal  results.  (2)  In  the  latter  case,  the  primary  source  of  initiative  is 
the  decision  aid,  which  implements  a  normative  approach  to  the  problem.  The 
aid,  however,  monitors  its  own  performance  for  weaknesses  (e.g.,  conflicting 
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lines  of  reasoning  or  incomplete  information)  and  prompts  the  user  when  it 
concludes  that  the  user  -is  likely  to  make  a  significant  contribution  to  the 
problem  and  user  workload  is  at  an  acceptable  level. 

Figure  2-5  outlines  some  of  the  characteristics  of  the  application  that  deter¬ 
mine  which  of  these  modes  should  prevail.  Typically,  primary  adaptation  to 
the  decision  maker  (and  greater  human  initiative)  is  appropriate  when  there  is 
relatively  low  time  stress,  users  are  relatively  high-level  decision  makers, 
and  the  task  is  relatively  "unstructured,"  i.e.,  options,  key  uncertainties, 
and/or  dimensions  of  value  are  to  some  degree  undefined.  Primary  adaptation 
to  the  situation  (and  greater  computer  initiative)  is  more  appropriate  in 
high- time  stress,  low-level,  structured  tasks.  This  distinction  corresponds 
to  the  predicted  dominance  of  knowledge -based  versus  stereotypical  perfor¬ 
mance  . 

Figure  2-6  outlines  the  design  steps  involved  in  Personalized  and  Prescriptive 
Decision  Support.  The  key  point  is  to  model  both  user  strategies  and  a 
relevant  normative  approach.  After  that,  the  specific  conditions  (if  any)  un¬ 
der  which  a  user's  approach  is  likely  to  be  suboptimal,  according  to  the  nor¬ 
mative  model,  can  be  identified.  At  the  same  time,  potential  advantages,  if 
any,  of  permitting  users  to  deal  with  the  problem  in  their  preferred  way  are 
noted.  The  choice  of  a  basic  aiding  mode  depends  on  the  features  discussed 
above  (degree  of  structure,  level  in  organization,  time  stress) ,  as  well  as  on 
the  results  of  the  preceding  steps.  Thus,  primary  adaptation  to  the  decision 
maker  presupposes  that  there  is  significant  value  in  exploiting  the  user's  un¬ 
aided  approach  to  the  problem  (this  is  more  likely  to  be  the  case  in  unstruc¬ 
tured  problems  under  low  time  stress).  Primary  adaptation  to  the  decision 
maker  also  presupposes  that  any  significant  biases  in  the  user's  approach  can 
be  identified,  and  that  the  conditions  of  their  occurrence  can  be  specified. 

When  primary  adaptation  is  to  the  decision  maker,  a  variety  of  prescriptive 
methods  may  be  selected  to  reduce  the  impact  of  biases.  Specifically,  as 
shown  in  Figure  2-7,  the  decision  aid  can  operate  in  either  a  proactive  or 
reactive  manner,  with  advisory  guidance  that  is  either  explicit  or  implicit. 
Guidance  is  proactive  if  it  is  incorporated  into  the  design  independently  of 
any  specific  evidence  for  biased  judgment  on  the  part  of  a  particular  decision 
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TASK  ALLOCATION  INVOLVES  DETERMINATION  OE 
BALANCE  OE  INITIATIVE  BETWEEN  HUMAN  AND 

COMPUTER. 


LOW  TIME  STRESS  HIGH  TIME  STRESS 

HIGH-LEVEL  IN  ORGANIZ.  LOW-LEVEL  IN  ORGANIZ. 

"UNSTRUCTURED"  TASK  "STRUCTURED"  TASK 


PRIMARY  ADAPTION 
TO  DECISION  MAKER; 

HUMAN  INITIATIVE; 
COMPUTER  MONITORS 
HUMAN  PERFORMANCE 
AND  PROVIDES  HELP 


PRIMARY  ADAPTATION 
TO  SITUATION; 
COMPUTER  INITIATIVE; 
COMPUTER  MONITORS  OWN 
PERFORMANCE  AND  ASKS 
FOR  HELP 


Figure  2-5:  Some  Factors  Involved  in  Determining  Allocation 
of  Cognitive  Tasks  Between  Computer  and  User 
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DESIGN  STEF 


USE: 


I.  IDENTIFY  POTENTIAL  USER  PREFERENCE 
IN  REPRESENTING  KNOWLEDGE  OR 
SOLVING  PROBLEM. 


COOMTM  tCZNCt  LJTDUTUnC 
OKMUDCE  {DOTATION 
CXRUJRATORY  CXRCTSCKT3 


II.  IDENTIFY  MOST  APPROPRIATE 
NORMATIVE  MODEL(S). 


At  0*.  OR.  ETC 


111.  IDENTIFY  CONDITIONS  OF  SUB- 
OPTIMALITY  IN  USER  APPROACH. 


MATHEMATICAL  COMPARISON 
WITH  HORUATNE  THEORY; 
COtVUKSON  OF  ALTD04ATOE 
NORUAJM:  THEORIES 


IV.  IDENTIFY  POTENTIAL  ADVANTAGES 
OF  USER  APPROACH. 


COCNmvE  SOEHCE  UICTATURE 
KHOWUXXX  {DOTATION 
CXPUXUTOmr  EXPEWUENTS 
awmoNs 


V.  CHOOSE  ALLOCATION  SCHEME:  (A) 
HUMAN  INITIATIVE  WITH  COMPUTER 
HELP,  (B)  COMPUTER  INITIATIVE  WITH 
HUMAN  HELP. 


FORM.  OR  NFORUN. 
KOOELS  OF  USER/SYSTEM 
PERFORMANCE 


VIA.  DESIGN  AJD  FEATURES  THAT  FACILITATE 
BASIC  USER-PREFERRED  METHOD.  BUT 
PROVIDE  PROTECTION  AGAINST  SPECIFIC 
IDENTIFIED  PITFALLS.  PROTECTION  MUST 
MESH  SUFFICIENTLY  WITH  PREFERRED 
APPROACH  SO  THAT  ITS  ADVANTAGES  ARE 
PRESERVED. 


TESTS  OF  WERALL  SYSTEM 
FtRFORUANCE:  UTliZATXJN; 
CONFKJCE  AMD  SATISFACTION 
OF  USER  -  AS  A  FUNCTION  OF 
SPECTK  AJD  FEATURES 


VIB.  DESIGN  AID  FEATURES  THAT  IMPLEMENT 
NORMATIVE  MODEL,  BUT  BRING  USER 
INTO  PROCESS  WHERE  HE  CAN  CONTRIBUTE. 
USER  INPUTS  MUST  MESH  WITH  PREFERRED 
USER  APPROACH  TO  PROBLEM,  AND  NOT 
DISRUPT  HIGHER  PRIORITY  TASKS. 


Figure  2-6:  Elements 
Decision  Support 


of  the  Personalized  and  Prescriptive 
Approach  to  Decision  Aid  Design 
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PRESCRIPTIVE  METHODS 


RECOMMENDED  USER  ACTION  IS: 


EXPLICIT 

IMPLICIT 

PROACTIVE 

INSTRUCTION 

CHANNELING 

CONTEXT  FAVORS  MORE 
OPTIMAL  VARIANT  OF  USER- 
PREFERRED  APPROACH 

PROMPTING 

REACTIVE 

RECOUUEHD  ACTIONS  WHICH 
MESH  WITH  BUT  REMEDY 
SHORTCOMINGS  W  USER- 
PREFERRED  APPROACH 

OUTCOME  FEEDBACK 

GUIDELINES  FOR  THE  SELECTION  OF  A  PRESCRIPTIVE  METHOD: 


BIAS  IS  A  RESULT  OF  A  WAY  OF  DESCRIBING  OR  PERCEIVING 
THE  PROBLEM - ►  CHANNELING 

BIAS  IS  A  RESULT  OF  ACTIONS  UNDER  VOLUNTARY  CONTROL  OF 
DECISION-MAKER  - ►  PROMPTING  OR  INSTRUCTION 

OCCURRENCE  OF  BIAS  IS  NOT  INEVITABLE  - ►  PROMPTING 

BEST  ACTION  NOT  KNOWN;  LEEWAY  FOR  TRIAL  AND 
ERROR  - ►  OUTCOME  FEEDBACK 


Figure  2-7:  Prescriptive  Methods  for  Countering  Potential 

User  Biases 


43 


maker.  Guidance  is  reactive  if  it  is  provided  in  response  to  specific  deci¬ 
sion  maker  actions  on  a  particular  occasion.  Explicit  guidance  occurs  when¬ 
ever  the  decision  aid  makes  an  explicit  recommendation  to  the  decision  maker 
regarding  his  or  her  decision-making  procedures.  Implicit  guidance  indirectly 
causes  modification  in  decision  making  procedures  by  changing  the  decision 
maker's  perception  of  the  problem  or  of  the  success  of  his  or  her  current  ap¬ 
proach.  Instruction  on  problem-solving  procedures  is  thus  a  form  of  explicit, 
proactive  guidance. 

Prompting  is  a  form  of  explicit,  reactive  guidance.  Prompting  occurs  when  the 
decision  aid  recommends  a  user  action  to  remedy  a  possible  shortcoming  in 
results  generated  by  the  user-preferred  decision  process.  For  instance,  sup¬ 
pose  we  have  a  decision  problem  where  the  user-preferred  approach  is  to  select 
a  minimum  risk  option  (Figure  2-8  gives  an  example  of  this  sort.)  The  re¬ 
search  reported  in  Lopes  (1981)  suggests  that  decision  makers  often  select  the 
option  which  does  best  in  worst-case  assumptions,  while  a  normative  approach 
dictates  selecting  the  option  with  the  highest  expected  value  across  all 
outcomes.  One  advantage  of  a  worst  case  approach  is  that  it  permits  the  user 
to  focus  on  concrete,  realizable  states  of  affairs  (which  can  be  modeled 
causally)  as  opposed  to  the  abstract,  non-realizable  average  or  expected 
value.  Prompting  would  occur  when  a  decision  aid  informed  the  decision  maker 
that  an  option  existed  which  is  slightly  more  risky  than  the  minimum  risk  op¬ 
tion  but  had  a  much  better  outcome  on  non-worst-case  assumptions.  Note  that 
the  prompt  does  not  require  the  decision  maker  to  abandon  altogether  his 
preferred  mode  of  processing  in  favor  of  a  normative  approach.  Rather  than 
requiring  him  to  think  in  abstract  terms  (i.e.,  to  compare  the  expected  values 
of  each  option) ,  the  prompt  recommends  a  procedure  that  meshes  naturally  with 
his  original  approach  (look  only  at  worst  outcomes) ,  but  expands  it  (to  draw 
his  attention  to  an  option  that  does  very  well  on  better  outcomes). 

In  the  PDS  approach  instruction  too  (if  it  is  utilized)  should  mesh  as  closely 
as  possible  with  the  user's  natural  approach,  rather  than  impose  an  altogether 
new  method  (e.g.,  instructing  such  users  to  consider  non-worst  case  outcomes 
is  consistent  with  PDS;  instructing  them  in  expected  utility  theory  is  not) 

(cf . ,  Lopes,  1982).  Prompting  may  be  preferable  to  instruction,  however,  if 
the  potential  bias  does  not  inevitably  occur  whenever  the  strategy  is  used. 
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I)  USER  PREFERENCES 

THERE  IS  EXPERIMENTAL  EVIDENCE  (LOPES.  1986)  THAT 
SOME  PEOPLE  PREFER  TO  COMPARE  OPTIONS  IN  TERMS 
OF  THEIR  ASSOCIATED  WORST  CASE  SCENARIOS.  OPTION 
WITH  THE  "LEAST  BAD"  WORST  CASE  IS  SELECTED.  (OTHER 
PEOPLE  COMPARE  OPTIONS  IN  TERMS  OF  ASSOCIATED  BEST 
CASE  SCENARIOS.) 


II)  NORMATIVE  MODEL 

INVOLVES  ASSESSMENT  OF  PROBABILITIES  AND  VALUES  OF 
EACH  OUTCOME  OF  EACH  OPTION,  COMBINATION  INTO  AN 
EXPECTED  UTILITY  SCORE  FOR  EACH  OPTION,  AND 
SELECTION  OF  OPTION  WITH  HIGHEST  SCORE. 


Figure  2-8:  Example  of  Personalized  and  Prescriptive  Approach: 
Decision  Making  under  Uncertainty 
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III)  POTENTIAL  PITFALLS  OF  USER  APPROACH 

MAY  REJECT  OPTIONS  WHICH  ARE  SLIGHTLY  INFERIOR 
ON  WORST  CASE  ASSUMPTION,  BUT  DO  BETTER  IN  OTHER 
CIRCUMSTANCE. 

IV)  POTENTIAL  ADVANTAGES 

PERMITS  A  MORE  INTUITIVE,  LESS  ABSTRACT  APPROACH; 
CONSISTENT  WITH  NEED  TO  ANTICIPATE  AND  PLAN 
CONCRETELY  FOR  SPECIFIC  SITUATIONS. 

NATURAL  JUSTIFICATION  IN  TERMS  OF  GUARANTEED 
MINIMUM  INCOME. 

V)  DETERMINE  MODE  OF  AIDING 

SELECT  HUMAN-INITIATIVE  MODE  (E.G.,  IF  THIS  IS  AID 
FOR  HIGH-LEVEL,  NON  TIME-STRESSED  OPERATIONAL 
PLANNING). 

VI)  AIDING  APPROACH 

PERSONALIZATION:  UNDER  UNCERTAINTY,  MAKE  DEFAULT 
DISPLAYS  CORRESPOND  TO  WORST  CASE  SITUATION. 

CHANNELING:  ALSO  MAKE  AVAILABLE  DISPLAYS 
CORRESPONDING  TO  OTHER  POSSIBLE  SITUATIONS,  AND 
TO  AGGREGATED  VALUES. 

PROMPTING: 

-  PROMPT  WHEN  AN  OPTION  IS  REJECTED  WHICH 
IS  SIGNIFICANTLY  BETTER  ON  NON-WORST 
CASE  ASSUMPTIONS. 

-  PROMPT  FOR  DEVELOPMENT  OF  CONTINGENCY 
PLANS  IF  INFORMATION  PERTAINING  TO 
UNCERTAINTY  MIGHT  BE  OBTAINED  LATER. 


Figure  2-8:  Example  of  Personalized  and  Prescriptive  Approach: 
Decision  Making  under  Uncertainty  (continued) 
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In  the  example  above,  the  user-preferred  (worst  case)  method  will  be  sig¬ 
nificantly  inferior  to  the  normative  (expected  valcue)  method  only  when  there 
is  an  option  that  does  very  well  on  non-worst  case  assumptions  but  poorly  in 
the  worst  case.  Thus,  the  user  need  be  bothered  by  a  prompt  only  when  it 
really  matters. 

Channeling  is  a  form  of  implicit,  proactive  guidance.  Some  types  of  user- 
preferred  decision  strategies  may  be  subject  to  very  predictable  biases  or 
shortcomings.  Channeling  involves  the  tailoring  of  displays  such  that  the 
decision  maker  may  be  less  subject  to  these  possible  shortcomings.  For  in¬ 
stance,  decision  makers  may  prefer  an  elimination-by-aspects  decision  strategy 
(i.e.,  sequentially  considering  a  series  of  problem  aspects  or  factors,  and 
rejecting  options  that  fail  to  meet  criteria  or  goals  on  each  factor),  as  op¬ 
posed  to  normative  methods  like  multiattribute  utility  theory,  which  require 
explicit  (and  highly  abstract)  assessments  of  the  relative  importance  of 
different  aspects.  An  advantage  of  such  a  strategy,  once  again,  is  concrete¬ 
ness  (e.g.,  causally  modeling  the  achievement  of  specific  goals  on  specific 
dimensions).  Yet  very  good  options  may  be  inappropriately  rejected  because 
they  fail  to  meet  criteria  on  some  factors  selected  for  analysis,  even  though 
they  perform  outstandingly  well  on  other  factors.  By  providing  displays  that 
help  users  apply  an  elimination-by- aspects  strategy  while  at  the  same  time 
comparing  options  on  a  variety  of  factors,  an  aid  may  help  retain  the  advan¬ 
tages  of  this  approach  while  guarding  against  its  dangers;  decision  makers 
will  be  less  likely  to  reject  options  on  the  basis  of  a  single  factor.  In  ef¬ 
fect,  displays  are  designed  so  as  to  provide  a  context  that  favors  the  use  of 
a  more  optimal  variant  of  the  user-preferred  decision  strategy. 

Finally,  providing  the  user  with  outcome  feedback  on  the  anticipated  results 
of  a  selected  decision'  is  a  form  of  implicit,  reactive  guidance.  Such 
guidance  is  implicit  because  it  leaves  the  user  with  the  responsibility  of 
discovering  an  emendation  of  his  or  her  current  procedure  which  will  yield 
better  performance  (i.e.,  more  satisfactory  feedback).  This  form  of  guidance 
is  thus,  essentially,  a  matter  of  trial  and  error,  and  may  be  extremely  valu¬ 
able  where  the  appropriate  adaptation  to  a  situation  cannot  be  anticipated. 

In  inference  problems,  "feedback"  may  consist  not  of  "ground  truth"  concerning 
the  correctness  of  a  conclusion,  but  the  extent  of  its  agreement  or  disagree - 
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ment  with  other  lines  of  reasoning.  This  type  of  feedback  can  also  be  based 
on  the  results  of  an  aid-internal  simulator,  where  simulation  runs  are 
selected  to  point  out  user-preferred  vs.  normative  decision  strategy  dif¬ 
ferences  . 

In  summary,  PDS  represents  a  form  of  mixed- initiative  adaptation  to  the  deci¬ 
sion  maker  and  decision  situation.  When  adaptation  to  the  decision  maker  is 
predominant,  the  decision  aid  design  anticipates  and  provides  for  the  possible 
strategies  used  by  decision  makers.  The  decision  maker  is  required  to  have 
enough  understanding  of  the  decision  aid  and  enough  understanding  of  him-  or 
herself  to  be  able  to  select  from  among  the  alternative  available  decision 
strategies.  Once  a  decision  strategy  is  selected,  the  decision  aid  adapts  its 
procedures  and  displays  according  to  its  internal  model  of  the  characteristics 
of  the  user-preferred  strategy.  In  effect,  the  aid  uses  an  internal  model  of 
the  selected  decision  strategy  as  the  major  component  of  the  model  of  the 
decision  maker.  This  model  is  compared  by  the  aid  to  the  results  of  a 
normative  model,  and  prompts  (or  other  forms  of  guidance)  are  provided  that 
help  adapt  the  decision  maker/decision  aid  system  to  the  situation.  These 
prompts  are  themselves  influenced  by  the  aid's  model  of  the  decision  maker, 
and  are  designed  to  mesh  closely  with  the  user-preferred  strategy.  Finally, 
the  user  has  the  choice  of  determining  how  to  respond  to  the  offered  guidance. 

When  adaptation  to  the  situation  is  predominant,  the  aid  utilizes  a  model  of 
its  own  capabilities  to  detect  potential  weaknesses  in  its  performance  and  a 
model  of  the  decision  maker's  capabilities  to  determine  when  and  if  to  prompt 
the  user  for  contributions.  A  model  of  the  user's  preferred  ways  of  repre¬ 
senting  information  is  utilized  to  determine  the  form  and  manner  in  which  in¬ 
puts  are  requested.  Finally,  the  user  may  decide  whether  and  how  to  respond 
to  computer  prompts . 
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3.0  PILOT  KNOWLEDGE  ELICITATION  AND  DESIGN  OF  DISPLAY  CONCEPTS 


In  this  section  we  review  the  application  of  personalized  and  prescriptive 
decision  support  to  the  design  of  interactive  displays  for  intelligent  in¬ 
flight  avionic  systems.  As  described  briefly  in  Section  1.3,  the  application 
of  that  methodology  proceeded  in  four  steps: 

o  Structured  interviews  of  pilots, 
o  Development  of  preliminary  prototype  displays . 
o  Evaluation  and  comments  on  prototype  displays  by  pilots, 
o  Revision  of  prototype  displays. 

In  principle,  the  last  step  could  be  followed  by  evaluation  of  the  revised 
prototype  displays,  additional  revision  of  the  displays,  further  evaluation, 
and  so  on,  until  a  fully  satisfactory  design  had  been  developed.  Within  the 
constraints  of  this  six-month  project,  however,  such  additional  iterations 
were  not  possible. 

The  strategy  of  this  section  will  be,  first,  to  discuss  the  knowledge  elicita¬ 
tion  and  evaluation  methodology  in  somewhat  more  detail ;  we  then  take  up 
several  major  topics  in  sequence:  display  of  uncertainty,  checking  the 
validity  of  data  sources,  and  hierarchical  representations.  Within  each  of 
these  topics,  we  will  discuss  the  results  of  each  step  in  the  application  of 
the  PDS  methodology. 

3 . 1  Method 

Structured  interviews .  As  described  briefly  in  Section  1.3,  the  first  stage 
of  knowledge  elicitation  involved  structured  interviews  with  three  pilots  (as 
a  group).  The  pilots  were  led  through  a  typical  strike  scenario,  in  which 
various  events  were  hypothesized  and  the  pilots  were  asked  how  they  would 
think  about  or  act  upon  those  events.  The  basic  strategy  of  these  interviews 
was  to  focus  initial  queries  on  elementary  objects,  events,  and  properties  and 
then  to  gradually  add  complexity,  e.g. ,  multiple  threats  and  uncertainties. 
Despite  this  structure,  pilots  were  encouraged  to  talk  freely  about  any  re¬ 
lated  topics.  The  interviews  were  recorded. 
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During  the  interviews  questions  were  asked  on  the  following  topics: 

o  Approaching  a  single  known  ground  threat  enroute  to  the  target 

o  Approaching  two  known  ground  threats  enroute  to  the  target 

o  How  one  thinks  about  own  aircraft  location,  and  the  impact  of  loca¬ 
tion  landmarks  (i.e.,  passing  a  way  point,  getting  closer  to  a  tar¬ 
get,  passing  a  threat) 

o  Comparing  the  relative  danger  of  two  threats 

o  Choosing  between  routes  which  differ  on  various  dimensions  (i.e., 

vulnerability  to  different  types  of  threats,  superiority  on  ingress 
versus  egress) 

o  Encountering  an  unexpected  ground  threat  enroute  to  the  target 

under  various  conditions  (i.e.,  with  or  without  high  density  of  sur¬ 
rounding  threats,  with  or  without  fuel  constraints,  with  or  without 
limited  chaff,  with  or  without  a  heavy  bomb  load,  and  with  or 
without  jamming  capability) 

o  Encountering  an  unexpected  air  threat  enroute  to  the  target  under 
various  conditions  (same  as  above) 

o  Encountering  unexpected  air  and  ground  threats  simultaneously 

o  Uncertainty  about  the  location  or  number  of  ground  threats  under 
various  conditions  (i.e.,  degree  of  overlap  with  planned  flight 
path,  reliability  of  sources  of  data) 

o  Uncertainty  about  the  classification  of  an  unexpected  ground  threat 
under  various  conditions  (i.e.,  impact  of  uncertainty  on  projected 
flight  path,  reliability  of  sources) 
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o  Choice  among  routes  which  differ  in  risk  (i.e.,  avoiding  an  uncer¬ 
tain  but  dangerous  threat  versus  avoiding  a  less  dangerous  but  known 
threat  versus  a  hedging  strategy) 

Prototype  development  and  evaluation .  The  next  steps  in  the  knowledge 
elicitation  process  involved  analysis  of  the  structured  interviews,  develop¬ 
ment  of  preliminary  prototype  displays ,  and  evaluation  of  those  displays  by 
pilots.  The  displays  were  implemented  on  an  IBM- PC/AT  which  presented  the 
displays  in  the  context  of  an  illustrative  ground  strike  scenario. 

Prototype  evaluation  consisted  of  two  phases,  conducted  individually  with  each 
of  the  three  pilots:  (a)  an  initial  run-through  of  the  sequence  of  displays 
in  the  sample  scenario  to  familiarize  the  pilot  with  the  scenario  and  with  the 
basic  features  of  the  prototype  system;  and  (2)  a  second  run-through  of  the 
sample  displays  with  comments  and  quantitative  evaluations.  Each  of  the  three 
pilots  was  asked  to  rate  twenty- four  specific  display  features  on  a  seven- 
point  scale  based  on  his  experience  with  current  cockpit  equipment.  1  indi¬ 
cated  "very  good,"  4  indicated  "neutral,"  and  7  indicated  "very  poor."  Com¬ 
ments  were  also  solicited  from  the  pilots  regarding  these  and  other  display 
features . 

A  final  version  of  prototype  display  system  was  then  developed  based  on  the 
pilot  evaluations .  Displays  for  that  prototype  system  are  presented  and 
described  in  the  Appendix  in  the  order  of  the  sample  scenario;  in  the 
remainder  of  this  section,  however,  they  will  be  discussed  in  the  context  of 
specific  topics  to  which  the  design  methodology  was  applied. 

3 . 2  Uncertainty 

Structured  interviews.  The  theory  of  mental  models  which  we  developed  in  Sec¬ 
tion  2.0  implies  that  decision  makers  in  general,  and  pilots  in  particular, 
should  experience  difficulty  simultaneously  considering  multiple  possible 
situations,  and  that  problem-solving  efforts  will  be  oriented  towards  arriving 
at  a  single  acceptable,  concrete  (i.e.,  analogical)  representation.  This 
hypothesis  was  confirm  id  in  the  structured  interviews  with  pilots.  The 
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interviews  revealed,  however,  that  a  variety  of  relatively  sophisticated 
methods  for  arriving  at  such  a  representation  are  utilized: 

o  If  sensors  confirm  the  presence  of  the  threat  but  are  inconclusive 
regarding  its  classification,  pilots  adopt  a  worst  case  assumption, 
i.e.  they  assume  that  the  threat  has  maximum  plausible  capability 
against  them.  The  rationale  for  this  assumption  is  that  the  failure 
to  classify  the  threat  is  itself  evidence  that  the  threat  is  a  new 
system,  and  therefore  likely  to  be  more  dangerous  than  previously 
known  threats . 

o  On  the  other  hand,  if  available  information  is  inadequate  to  confirm 
the  existence  of  a  threat,  pilots  tend  to  make  a  best  case  assump¬ 
tion,  i.e.,  they  assume  that  the  threat  is  not  present  until  more 
definite  information  is  obtained.  The  rationale  for  this  assumption 
is  that  actions  taken  to  avoid  the  threat  would  almost  inevitably 
expose  the  aircraft  to  risk  from  other  known  threats.  Nevertheless, 
even  in  this  situation,  limited  action,  e.g.,  speeding  up  the 
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aircraft,  might  be  taken  to  reduce  risk  from  the  unconfirmed  threat. 

o  Even  when  the  existence,  location,  and  presence  of  a  threat  is  known 
in  advance,  there  may  be  uncertainty  about  its  actual  capabilities. 
Pre-briefed  intelligence  generally  focuses  on  maximum  capabilities, 
disregarding  degradation  during  the  course  of  combat.  Pilots,  on 
the  other  hand,  assume  that  in  practice  all  systems  are  subject  to  a 
significant  amount  of  degradation;  as  a  result  they  tend  to  apply  a 
general  discounting  factor  to  the  threat  as  assessed  in  intelligence 
reports. 

Prototype  displays .  Based  on  the  results  of  the  structured  interview, 
prototype  displays  were  designed  satisfying  the  implied  constraints  of  mental 
model  theory.  That  is,  displays  under  conditions  of  uncertainty  regarding 
threat  existence,  location,  or  classification  portrayed  single  possible  situa¬ 
tions  in  preference  to  probabilistic  averages.  The  particular  situation 
depicted,  however,  depended  on  the  type  of  uncertainty:  worst  case  displays 
for  classification  uncertainty  and  best  case  displays  for  existence/location 
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uncertainty.  As  a  partial  safeguard  against  focusing  exclusively  on  a  single 
possibility,  however,  displays  for  other  possibilities  as  well  as  an  ag¬ 
gregated  display  were  also  made  available . 

Figure  A-l  shows  the  first  screen  in  the  simulated  ground  strike  scenario. 

The  dotted  yellow  line  at  the  bottom  right  represents  the  FEBA;  the  blue 
aircraft  symbol  represents  own  aircraft;  the  solid  blue  line  represents  the 
planned  aircraft  route;  and  the  yellow  "T"  represents  the  target.  Ground 
threats  are  represented  by  generic  symbols  for  surface-to-air  missiles,  anti¬ 
air  artillery,  and  radar.  Different  shades  of  red  indicate  different  levels 
of  threat  to  the  aircraft  in  those  regions.  Figure  A- 7  indicates  a  later 
point  in  time  in  the  scenario  when  new  threat  information  has  been  received 
from  an  AWACS  (e.g.,  through  a  JTIDS  digital  data  link).  This  information 
suggests  the  possible  existence  of  a  new  threat  at  the  location  indicated  by 
the  yellow  lethality  contour.  In  this  scenario,  however,  interpretation  of 
that  data  is  uncertain:  it  could  indicate  the  existence  of  a  new  threat;  al¬ 
ternatively  the  AWACS  data  could  represent  a  previously  identified  threat 
which  has  changed  location  or  which  was  previously  mislocated.  Three  dif¬ 
ferent  displays  were  designed  to  represent  this  situation: 

o  The  worst  case  display  (Figure  A-7)  indicating  a  new  threat  on  the 
planned  route . 

o  A  best  case  display  (Figure  A- 8)  in  which  the  new  data  are  inter¬ 
preted  as  originating  from  a  previously  identified  threat,  and  are 
utilized  to  update  the  localization  of  that  threat.  (This  was  the 
default  display  in  this  scenario.) 

o  An  aggregated  or  average  display  (Figure  A-9)  in  which  the  lethality 
to  own  aircraft  at  any  given  point  is  computed  as  a  probability 
weighted  average  of  the  two  above  mentioned  possibilities. 

Some  common  features  of  all  three  displays  should  be  noted: 

o  Yellow  contours  are  utilized  to  represent  the  receipt  of  new  infor¬ 
mation  which  increases  estimated  danger  to  the  aircraft.  For  each 
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display,  regions  are  shaded  in  yellow  when  the  increase  in  danger  in 
that  region,  based  on  the  new  information,  exceeds  a  pre-specified 
threshold  (e.g.,  twenty  percent).  Estimated  increments  in  danger  to 
the  aircraft  are  based  on  worst  case  and  best  case  assumptions  in 
displays  A- 7  and  A- 9,  respectively. 

o  Uncertainty  is  represented  by  the  association  of  the  red  SAM  symbol 
in  the  displays  with  a  red  question  mark. 

As  shown  in  Figures  A- 10  and  A- 11,  pilots  were  able  to  request  a  recommended 
route  revision  which  took  into  account  the  new  threat  information.  The  recom¬ 
mended  revision  could  be  requested  in  the  context  of  any  of  the  three 
displays:  i.e.,  a  route  revision  based  on  the  worst  case  assumption  (Figure 

A-10) ,  a  route  revision  based  on  the  best  case  assumption,  or  a  route  revision 
based  on  the  probabilistic  average  (Figure  A- 11) . 

A  similar  set  of  displays  was  prepared  to  represent  uncertainty  about  the 
classification  of  a  threat.  Figure  A- 30  represents  own  aircraft  having  passed 
the  target  and  beginning  the  egress  phase  of  the  mission.  In  Figure  A-31  on¬ 
board  EW  equipment  suggests  that  a  threat  previously  classified  as  an  SA-2  may 
in  fact  be  an  SA-4.  Figure  A-31  shows  the  worst  case  assumption:  that  the 
new  threat  is  an  SA-4  (note  that  these  threat  contours  are  entirely  fictional, 
and  bear  no  relation  to  actual  threat  capabilities) .  Again  the  yellow  regions 
indicate  areas  where  danger  to  own  aircraft  would  be  increased  by  a  given  per¬ 
centage  on  the  assumption  that  the  threat  is  an  SA-4.  Figure  A- 32  represents 
the  best  case  assumption:  i.e.,  that  the  threat  is  an  SA-2,  as  previously 
believed.  Finally,  Figure  A- 33  represents  a  probabilistic  average  of  the  two 
possibilities.  In  this  context,  the  worst  case  display  (Figure  A-31)  was  the 
default. 

Evaluation.  Our  hypothesis,  based  on  the  theory  of  mental  models  and  on  the 
results  of  our  structured  interview,  was  that  pilots  would  prefer  single  pos¬ 
sibility  displays  (e.g.,  worst  case  or  best  case)  to  probabilistically 
averaged  displays.  In  addition,  we  had  a  less  strong  prediction  regarding 
which  of  the  two  single  possibility  displays  would  be  preferred:  worst  case 
displays  in  the  case  of  uncertainty  about  threat  classification,  and  best  case 
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displays  in  the  case  of  uncertainty  about  threat  existence/location.  Finally, 
we  proposed  a  prescriptive  counterbalance  against  the  likelihood  that  pilots 
would  focus  exclusively  on  single  possibility  displays.  The  menu  options  for 
the  display  of  other  possibilities  and  for  the  display  of  a  probabilistic  ag¬ 
gregation  constitute  a  "channeling"  device  (Section  2.4)  which  encourages  more 
optimal  sampling  of  information.  Since  this  is  explicitly  intended  as  a  coun¬ 
terbalance  to  the  pilot's  tendency  to  focus  on  single  possibilities,  we  did 
not  predict  strongly  favorable  responses  from  pilots.  Nevertheless,  we  would 
expect  that  the  display  of  other  possibilities  would  conform  more  closely  to 
pilot  mental  models,  hence,  be  somewhat  preferable  to  the  option  of  viewing  an 
aggregated  display. 

In  the  pilot  evaluation  of  the  prototype  system,  our  main  hypothesis  was . 
strongly  confirmed.  Pilots  strongly  preferred  automatic  presentation  of  dis¬ 
plays  of  specific  possible  situations  (e.g.,  assuming  a  particular  threat 
location  or  threat  classification)  to  probabilistically  aggregated  displays. 
The  following  table  gives  the  quantitative  evaluations  of  this  display 
feature : 

Presentation  of  specific  nossibilities 

Existence/location  uncertainty  212 

Classification  uncertainty  212 

In  this  (as  in  all  subsequent  tables),  the  three  columns  correspond  to  the 
three  pilots  who  participated  in  the  evaluation.  The  pilot  represented  in  the 
far  right  column  was  more  senior  than  the  other  two. 

However,  our  secondary  hypothesis,  regarding  which  automatically  provided 
single  possibility  displays  would  be  preferred  under  different  conditions,  was 
only  partially  confirmed.  For  uncertainty  regarding  threat  classification, 
pilots  did  indeed  prefer  worst  case  displays.  However  they  also  preferred 
worst  case  displays  when  uncertainty  pertained  to  the  existence/location  of 
the  threat: 

Presentation  of  best  case 

Existence/location  uncertainty  676 
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Presentation  of  worst  case 


Classification  uncertainty  212 

Pilots  were  then  queried  regarding  the  option  of  being  able  to  see  the  other 
single  possibility  case.  As  might  be  expected,  given  its  introduction  as  a 
counterbalance  to  the  tendency  to  use  only  a  single  possibility,  the  pilots 
were  mixed  (mildly  opposed,  neutral,  mildly  favorable)  in  their  evaluation  of 
this  option: 

Presentation  of  other  possibility 

Existence/location  uncertainty  543 

Classification  uncertainty  5  3  3 

They  were  more  mixed  (mildly  favorable  to  strongly  opposed)  in  their  evalua¬ 
tion  of  the  option  of  seeing  a  probabilistic  aggregation: 

Presentation  of  average 

Existence/location  uncertainty  376 

Classification  uncertainty  365 

Two  of  the  three  pilots  thus  felt  they  were  more  likely  to  use  a  display  of 
the  other  concrete  possibility  than  a  display  of  the  probabilistic  average. 
Comments  by  these  two  pilots  supported  a  mental  model  interpretation  of  the 
results.  These  pilots  indicated  that  an  aggregated  display  would  be  so 
homogenized  as  to  be  meaningless,  and  were  confident  in  their  own  ability  to 
extract  any  relevant  lessons  by  switching  back  and  forth  between  the  two  con¬ 
crete  displays. 

These  results  suggest  individual  differences  in  the  type  of  prescriptive  chan¬ 
neling  that  most  suits  pilots:  i.e.,  other  single  possibility  displays  versus 
probabilistic  averages.  The  most  important  result,  however,  is  that  one  or 
the  other  of  these  options  was  acceptable  (favorable  or  neutral)  to  all  the 
pilots,  and  thus  might  be  expected  to  function  effectively  as  a  prescriptive 
counterbalance . 
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Pilots  were  strongly  favorable  in  their  evaluation  of  the  color  coded  indica¬ 
tion  of  increased  danger  due  to  new  threat  information,  i.e.,  yellow  contours: 

Color-coded  indication  of  increased  danger  212 

However,  pilots  also  expressed  a  need  for  an  additional,  auditory  warning  in 
these  circumstances. 

Pilots  were  also  strongly  favorable  in  their  evaluation  of  the  system's 
capability  of  providing  a  recommended  route  revision  to  accommodate  new  threat 
information: 

Recommended  route  revision  112 

However  they  were  strongly  mixed  when  asked  whether  route  recommendations 
should  be  provided  at  the  pilot's  request  (as  in  our  prototype)  or 
automatically : 

Routes  -provided  at  pilot  request  16  5 

One  pilot  strongly  preferred  that  such  recommendations  be  provided  only  at  the 
pilot's  request,  while  the  other  two  pilots  had  reasonably  strong  preferences 
for  the  automatic  provision  of  such  recommendations.  Again,  the  data  suggest 
individual  differences,  which  could  perhaps  be  accommodated  in  a  final  system. 

Prototype  System  Revision .  These  data  provide  support  for  both  the  personal¬ 
ized  and  prescriptive  aspects  of  PDS  (Section  2.4).  The  effort  to  tailor  dis¬ 
plays  to  user-preferred  methods  of  representing  knowledge  and  solving  problems 
was  successfully  accomplished  by  means  of  the  theory  of  mental  models,  accord¬ 
ing  to  which  pilots  prefer  automatic  presentation  of  single  possibility  dis¬ 
plays  in  the  context  of  uncertainty.  The  prescriptive  aspect  of  this  system 
guards  against  the  tendency  to  focus  exclusively  on  such  a  display,  by  provid¬ 
ing  users  with  the  option  of  viewing  either  other  single  possibility  displays 
or  a  probabilistic  aggregation.  One  or  the  other  of  these  two  prescriptive 
options  proved  to  be  acceptable  to  all  of  the  pilots . 
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Nevertheless,  the  choice  of  which  single  possibility  display  to  present  under 
what  circumstances  proved  more  complex  than  we  anticipated.  Additional  ex¬ 
perimentation  and  iterations  of  the  prototype  system  would  be  required  to 
fully  explore  this  question.  A  plausible  hypothesis,  however,  based  on  the 
initial  structured  interview  as  well  as  pilot  comments  during  the  evaluation 
session,  is  the  following: 

In  cases  of  conflict  of  evidence,  i.e.,  where  there  are  plausible  argu¬ 
ments  on  both  sides,  pilots  consistently  adopt  the  worst  case  assumption. 
This  applies  whether  uncertainty  pertains  to  location/existence  or  class¬ 
ification  of  a  threat.  On  the  other  hand,  in  cases  where  evidence  is  in¬ 
complete,  i.e.,  the  available  evidence  points  in  one  particular  direction 
but  is  insufficiently  reliable  to  substantiate  that  possibility,  pilots 
have  a  greater  tendency  to  adopt  a  best  case  assumption.  In  particular, 
best  case  assumptions  will  be  favored  if  actions  based  on  the  worst  case 
possibility  are  associated  with  known  cost  (i.e.,  increased  risk  from 
other,  known  threats). 

In  the  final  version  of  the  prototype  system,  displays  were  designed  to 
reflect  this  hypothesis.  Thus  in  both  of  the  conflict  situations  described 
above  (uncertainty  about  location/existence  and  uncertainty  about 
classification) ,  the  default  display  provided  to  the  pilot  represented  the 
worst  case,  while  the  pilot  had  the  option  of  viewing  the  best  case  or  the  ag¬ 
gregated  display.  In  addition,  however,  we  created  another  situation,  earlier 
in  the  mission,  to  represent  incompleteness  of  evidence.  Thus  in  Figure  A- 2 
the  aircraft  has  received  a  message  by  electronic  data  link  from  the  AWAGS 
suggesting  the  possible  existence  of  a  threat  on  its  route.  Since  this 
evidence  is  regarded  as  insufficiently  reliable  on  its  own  to  establish  the 
existence  of  such  a  threat,  and  has  not  as  yet  been  confirmed  by  any  other 
data  source,  the  system  adopts  a  modified  best  case  assumption.  The  possible 
existence  of  the  threat  is  indicated  by  an  empty  yellow  contour  line  and  a 
question  mark.  If  he  wishes,  the  pilot  may  also  view  the  worst  case  pos¬ 
sibility,  as  shown  in  Figure  A- 3.  A  few  moments  later  in  this  scenario,  the 
existence  of  a  new  threat  is  confirmed  by  on-board  radar.  As  shown  in  Figure 
A-4,  when  this  occurs,  the  inference  mechanism  in  the  system  regards  the  ex¬ 
istence  of  a  new  threat  as  established  and  displays  to  the  user  reflect  that 
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conclusion.  In  Figure  A-5  the  pilot  has  requested  a  recommended  route  revi¬ 
sion  based  on  the  existence  of  such  a  new  threat. 

3 . 3  Validity  Checking  of  Data  Sources 

In  order  to  arrive  at  a  single  concrete  representation  of  an  ambiguous  state 
of  affairs,  pilots  must  engage  in  relatively  sophisticated  processes  of 
problem  solving.  Such  processes  were  touched  on  earlier  in  our  discussion  of 
mental  models  (Section  2.2):  both  deKleer  and  Brown  and  Johnson-Laird  focused 
on  the  use  of  assumptions  to  derive  a  concrete,  analogical  representation.  In 
Section  3.1  above  we  confirmed  that  pilots  engage  in  processes  of  this  sort. 
For  example,  when  evidence  for  the  existence  of  the  threat  is  incomplete,  and 
avoiding  the  threat  would  incur  risk,  then  pilots  assume  the  threat  does . not 
exist.  When  there  is  conflicting  evidence,  i.e.,  evidence  pointing  in  both  of 
two  directions,  we  saw  that  pilots  tend  to  assume  that  the  situation  with 
greatest  impact  ontheir  mission,  i.e.,  the  worst  case,  is  true. 

More  active  problem  solving  strategies  are,  however,  available  to  the  pilot. 
When  evidence  is  incomplete,  he  may  actively  seek  additional  confirming  data. 
When  evidence  is  conflicting,  he  may  search  for  an  explanation  of  the  conflict 
and  actively  seek  to  resolve  it  by  revising  assumptions  about  the  sources  of 
data.  Our  hypothesis  regarding  these  more  active  processes  is  based  on  the 
theory  of  mental  models  laid  out  in  Section  2.2  above:  that  pilot  problem¬ 
solving  strategies  for  dealing  with  incomplete  or  conflicting  data  will  util¬ 
ize  concrete,  causal  models  of  sources  of  data  and  of  the  factors  which  might 
enhance  or  interfere  with  their  accuracy. 

Structured  interviews .  The  structured  interviews  dramatically  confirmed  this 
hypothesis.  While  in  flight  over  enemy  territory,  pilots  do  not  simply  accept 
pre-briefed  intelligence  regarding  threat  locations  and  classifications. 
Rather,  they  use  such  intelligence  as  a  fallible  guide  in  an  active  process  of 
seeking  additional  information.  In  this  process,  the  pilot  continuously 
cross -validates  information  from  his  own  sensors  and  from  communications 
sources  with  prior  expectations  based  on  pre-briefed  intelligence.  When  data 
sources  do  not  agree,  moreover,  the  pilot  calls  upon  his  causal  understanding 
of  the  factors  that  affect  each  source  in  order  to  adjudicate  the  conflict. 
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For  example,  a  major  source  of  uncertainty  with  respect  to  pre-briefed  intel¬ 
ligence  is  the  mobility  of  SAM  sites.  Such  mobility  is  greater  close  to  the 
FEBA  than  it  is  deep  within  enemy  territory.  Therefore,  other  things  being 
equal,  the  credibility  of  pre-briefed  intelligence  relative  to  other  sources 
of  data  will  be  greater  during  deep  penetration  phases  of  the  mission.  In 
general,  pilots  attach  more  credibility  to  more  recent  in-flight  information 
which  is  received  from  friendly  returning  aircraft,  AWACS ,  ABCC  aircraft,  or 
own  sensors.  These  sources,  however,  are  also  subject  to  error:  for  example, 
radar  data  may  be  affected  by  ground  reflectance,  weather,  or  electronic  coun¬ 
termeasures.  The  pilot  himself  will  often  be  in  a  position  to  verify,  either 
visually  or  through  instruments,  whether  any  of  these  conditions  obtain  and 
will  evaluate  data  sources  accordingly.  This  continual  process  of  re- 
evaluation  and  cross  validation  may  not  only  provide  a  resolution  of  the  im¬ 
mediate  conflict,  but  also  provides  a  longer  term  cumulative  assessment  of  the 
credibility  of  the  various  information  sources.  For  example,  repeated  failure 
to  confirm  RHAW  scope  warnings  (of  illumination  by  a  threat)  through  other 
data  sources  may  lead  pilots  to  disregard  or  even  turn  off  that  piece  of 
equipment . 

In  accordance  with  the  theory  of  mental  models,  this  problem-solving  process 
is  causal  and  qualitative  rather  than  numerical  and  statistical.  In  the  in¬ 
terview  pilots  made  it  clear  that  they  did  not  wish  to  think  about  uncertainty 
in  a  numerical  fashion. 

Prototype  displays .  Conflict  of  evidence  represents  an  anomalous  (although 
not  altogether  infrequent)  situation  which  often  leads  to  knowledge -based 
reasoning  on  the  part  of  the  pilot.  Such  reasoning,  and  the  construction  and 
manipulation  of  mental  models  which  it  entails,  demands  considerable  cognitive 
effort.  The  aim  of  personalized  and  prescriptive  decision  support  (Section 
2.5)  in  this  context  is  to  automate  aspects  of  this  reasoning  process  which 
can  be  adequately  taken  over  by  a  computer,  while  continuing  to  tap  the 
pilot's  knowledge  and  judgment  only  on  those  occasions  where  he  can  uniquely 
and  significantly  contribute  to  a  solution.  Moreover,  to  maximize  the  pilot's 
contribution,  displays  should  be  designed  which  are  compatible  with  his  causal 
mental  models  of  the  data  sources  and  which  relieve  some  of  the  burden  on 
memory  and  computation  involved  in  constructing  and  running  such  models. 
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A  set  of  prototype  displays  was  developed  with  these  objectives  in  mind.  They 
support  the  pilot  both  under  conditions  of  incomplete  evidence,  where  he  must 
actively  search  for  additional  data,  and  under  conditions  of  conflicting 
evidence,  where  he  must  actively  search  for  conditions  that  would  causally 
discredit  one  or  more  of  the  pre-existing  data  sources. 

Figures  A-2  through  A-6  illustrate  the  function  of  these  displays  under  condi¬ 
tions  of  incomplete  evidence.  (As  noted  in  Section  3.2,  these  particular 
screens  were  developed  for  the  final  version  of  the  prototype  system,  and  were 
not  provided  specifically  to  the  pilots  for  evaluation.)  In  these  screens 
each  potential  source  of  data  regarding  a  threat  is  graphically  represented  by 
an  icon.  Thus  on  the  left  side  of  Figure  A-2,  from  top  to  bottom,  the  folder 
represents  pre-briefed  intelligence,  the  aircraft  stands  for  on-board  sensors 
or  pilot  visual  observation,  and  the  lightning  bolt  stands  for  communications 
from  air  or  ground  stations.  What  a  source  of  data  has  to  say  about  a  par¬ 
ticular  threat  is  represented  by  its  color:  a  green  icon  means  that  the  cor¬ 
responding  data  source  supports  the  best  case  possibility;  a  red  icon  means 
that  the  corresponding  data  source  supports  the  worst  case  possibility; 
finally,  a  blank  icon  means  that  no  reliable  data  has  been  obtained  from  that 
source.  The  essential  idea,  therefore,  is  to  enable  the  pilot  to  see  at  a 
glance  how  much  support  there  is  for  a  particular  possibility  and  where  that 
support  is  coming  from. 

In  Figure  A-2,  for  example,  the  red  lightning  bolt  indicates  that  the  data 
link  source  (i.e.,  the  AWACS)  supports  the  existence  of  a  new  threat  along  the 
planned  route;  the  blank  icons,  however,  indicate  that  this  data  is  not 
confirmed:  pre-briefed  intelligence  and  on-board  sensors  respectively  have 

provided  no  reliable  information  on  the  presence  or  absence  of  this  threat. 
Figure  A-2  is  designed  to  make  all  this  information  visually  accessible  to  the 
pilot  in  an  instant.  In  Figure  A-4  the  data  source  icon  representing  own 
aircraft  sensors  has  turned  red.  This  visual  cue,  accompanied  by  an  auditory 
alert,  immediately  informs  the  pilot  that  the  initial  report  of  a  new  threat 
has  been  confirmed. 
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Figures  A- 7  through  A- 16  (which  were  provided  to  the  pilots  for  evaluation) 
illustrate  the  function  of  this  display  design  under  conditions  of  conflicting 
evidence.  In  Figure  A- 7  data  sources  point  to  two  different  possibilities: 
either  there  is  an  unexpected  surface-to-air  missile  site  along  the  planned 
route  of  the  aircraft,  or  a  previously  identified  threat  has  moved  or  was  pre¬ 
viously  mislocalized.  By  glancing  at  the  iconic  display,  the  pilot  can 
quickly  diagnose  the  extent  and  nature  of  the  conflict  among  data  sources.  An 
iconic  display  in  which  all  icons  were  green  or  red  would  indicate  complete 
agreement.  In  this  case,  the  nearly  equal  mix  of  red  and  green  reflects  ex¬ 
treme  disagreement.  The  AWACS ,  represented  by  the  red  lightning  bolt,  sup¬ 
ports  the  existence  of  the  new  threat;  pre -briefed  intelligence,  represented 
by  the  green  folder,  supports  the  view  that  no  new  missile  sites  have  been  in¬ 
troduced  into  the  area.  Own  aircraft  sensor  information,  represented  by  the 
red  and  green  aircraft  symbol,  is  consistent  with  both  possibilities.  In  ad¬ 
dition,  an  explicit  verbal  indicator  of  "CONFLICT"  is  also  provided. 

The  iconic  displays  do  more  than  simply  inform  the  pilot,  in  a  visually  im¬ 
mediate  manner,  about  the  current  situation;  they  also  enable  him  to  con¬ 
tribute  his  own  knowledge  in  an  active  way  to  the  resolution  of  the  conflict. 
This  capability  is  made  possible  by  an  inference  mechanism  described  in  Cohen 
et  al.  (1986b).  That  inference  mechanism  differs  in  a  significant  way  from 
standard  normative  approaches  (e.g.,  Bayesian  probability,  Shaferian  belief 
functions,  or  fuzzy  logic)  in  its  treatment  of  conflict.  Rather  than  numeri¬ 
cally  aggregating  divergent  sources  of  information,  it  initiates  a  process  of 
heuristic  reasoning  which  attempts  to  determine  and  correct  the  cause  of  the 
conflict.  It  thus  interprets  conflict  among  data  sources  as  a  symptom  of  er¬ 
roneous  assumptions  regarding  the  validity  of  one  or  more  of  those  sources. 

The  system  attempts  to  resolve  the  conflict  by  selectively  revising 
assumptions- -collecting  additional  data  to  confirm  or  disconfirm  such  assump¬ 
tions  where  possible. 

Collection  of  additional  data  to  resolve  conflict,  e.g.,  through  deployment  of 
on-board  sensors  or  through  communication  with  other  ground  or  air  stations, 
is  determined  by  an  automatic  process  which  weighs  the  benefits  against  the 
costs  of  doing  so.  In  the  present  displays,  this  data  collection  process  has 
been  augmented  to  include  an  interactive  capability  for  tapping  the  knowledge 
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of  the  pilot.  Thus  if  resolution  of  the  conflict  among  competing  data  sources 
is  significant  for  mission  success  or  aircraft  safety,  if  the  pilot  is  likely 
to  possess  information  which  might  help  in  the  resolution  of  that  conflict, 
and  if  pilot  workload  is  at  an  acceptable  level,  then  the  system  may  query  the 
pilot  regarding  factors  that  would  potentially  discredit  one  or  more  of  the 
data  sources.  For  example,  in  Figure  A-13  the  system  has  asked  the  pilot 
whether  the  presence  of  electronic  countermeasures,  which  would  invalidate  the 
AWACS  evidence,  is  likely.  The  pilot  may  respond  to  this  query,  ignore  it,  or 
indicate  "no  information."  In  the  latter  case,  the  system  will  utilize  other 
methods  for  resolving  the  conflict,  possibly  including  another  query  to  the 
pilot . 

In  Figure  A- 14,  the  pilot  has  responded  to  the  query  by  indicating  that  ECM 
affecting  AWACS  is  indeed  a  problem;  the  icon  representing  the  AWACS  evidence 
has  changed  from  red  to  blank;  and  the  conflict  has  been  resolved. 

Evaluation .  Two  of  the  three  pilots  regarded  the  use  of  colored  icons  to  rep¬ 
resent  agreement  and  disagreement  among  data  sources  favorably,  while  one 
pilot  was  mildly  unfavorable: 

Icons  representing  conflict  532 

It  should  be  noted  that  the  most  experienced  pilot  was  also  the  most  favorable 
in  his  judgment  of  this  display.  A  further  (unscientific)  observation  is  that 
approval  was  correlated  with  the  order  in  which  the  pilots  were  exposed  to  the 
prototype  system;  we  suspect  our  own  skills  in  explaining  the  meaning  of  the 
iconic  display  improved  with  practice . 

Two  of  the  three  pilots  (although  a  different  two)  responded  favorably  to  the 
use  of  a  blank  icon  to  represent  a  data  source  which  has  been  discredited  in 
the  process  of  conflict  resolution: 

Blank  icon  for  discredited  source  343 

On  the  other  hand,  pilots  strongly  approved  the  explicit  indicator  of  conflict 
among  data  sources  (i.e.,  the  word  "CONFLICT"  in  yellow): 
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Indicator  of  conflict 
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It  should  be  noted  that  the  most  senior  of  the  three  pilots,  who  had  been  most 
favorable  toward  the  colored  iconic  representation  of  conflict,  was  the  least 
favorable  toward  the  explicit  verbal  indicator. 

The  pilots  were  also  favorable  towards  the  opportunity  to  provide  their  own 
judgmental  inputs  for  the  conflict  resolution  process: 

Judgmental  inputs  223 

Querying  of  the  pilot  by  the  system  however  was  acceptable  only  on  the  condi¬ 
tion  (a)  that  the  pilot  was  not  compelled  to  respond,  and  (b)  that  such 
queries  would  only  occur  when  the  problem  was  really  important.  One  pilot 
(the  most  senior)  expressed  an  interest  in  the  ability  to  directly  adjust  the 
credibility  of  a  data  source,  rather  than  indirectly  through  responses  to  sys¬ 
tem  queries . 

All  pilots  were  strongly  in  favor  of  an  automated  sensor  management  capability 
to  guide  the  collection  of  additional  data  for  the  resolution  of  conflict: 

Automated  sensor  management  212 

The  pilots  felt  that  the  pilot  should  be  queried  for  permission  to  redeploy  a 
sensor  only  when  the  pilot  himself  was  currently  utilizing  the  sensor  to  be 
redeployed. 

Final  prototype  system'.  These  results,  taken  as  a  whole,  support  the 
hypothesis  that  pilots  deal  with  uncertainty  by  utilizing  mental  models  of  the 
sources  of  data  and  that  displays  which  graphically  represent  what  those  data 
sources  have  to  say  can  effectively  support  pilots  in  that  process.  We  felt, 
however,  that  a  more  acceptable  introduction  to  the  iconic  displays  could  be 
provided  by  a  screen  which  was  less  complex  than  Figure  A-7  or  Figure  A-13. 
This  provided  another  motivation,  in  addition  to  those  discussed  in  Section 
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3.1  above,  for  introducing  Figures  A- 2  through  A- 6  into  the  final  prototype 
system. 


The  ultimate  objective  of  these  displays  is  to  provide  a  means  whereby  pilot 
knowledge  can  be  effectively  tapped  without  excessively  burdening  the  pilot  or 
delaying  the  system  response.  The  success  of  these  displays  in  that  regard 
must  eventually  be  evaluated  in  more  rigorous  empirical  tests.  Nevertheless, 
the  pilots  themselves  responded  quite  enthusiastically  to  the  opportunity  to 
insert  their  own  judgments  in  the  conflict  resolution  process.  In  the  final 
version  of  the  prototype  system,  this  capability  was  extended  somewhat  to  per¬ 
mit  the  pilots  to  discredit  a  data  source  directly  (by  pointing  and  clicking 
on  the  relevant  icon) ,  in  addition  to  indirectly  discrediting  it  by  responding 
to  system  queries. 

3 . 4  Hierarchical  Knowledge  Representation 

Pilots  must  of  necessity  think  about  their  mission  on  a  variety  of  levels.  In 
planning,  for  example,  they  must  keep  in  mind  the  overall  objectives  of  arriv¬ 
ing  at  the  target  with  the  required  ordnance  by  the  designated  time  and 
returning  safely  with  the  aircraft;  a  route  is  designed  which,  taken  as  a 
whole,  is  expected  to  achieve  those  objectives.  In  flight,  on  the  other  hand, 
the  pilot's  horizon  of  attention  may  expand  or  contract  radically,  depending 
on  the  circumstances.  On  occasions,  his  primary  concern  may  be  arriving  at 
the  next  way-point  at  the  appropriate  time;  at  other  times  his  only  concern 
may  be  the  immediate  evasion  of  an  active  threat;  on  still  other  occasions,  he 
may  need  to  balance  speed  versus  safety  in  replanning  a  significant  portion  of 
his  route  in  the  face  of  new  information.  The  hypothesis  to  be  investigated 
here  is  two-fold:  (1)  that  displays  should  be  appropriate  to  the  "world"  in 
which  the  pilot  is  currently  operating,  and  (2)  the  transition  from  one 
"world"  into  another  may  be  facilitated  by  providing  displays  that  are  (a) 
mutually  consistent  and  which  (b)  can  be  continuously  transformed  from  one 
into  the  other . 

Structured  interviews .  Pilots  think  of  their  world  from  two  extreme  points  of 
view,  corresponding  roughly  to  altitude.  Other  contrasts,  which  also 
characterize  pilot  knowledge  representations,  were  mentioned  during  the 
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interviews  (e.g.,  between  planning  and  flying;  ingress  and  egress).  But  the 
two  extremes  based  on  altitude  were  particularly  significant,  and  the  transi¬ 
tion  between  them  particularly  difficult;  a  description  of  them  will  suffice 
to  illustrate  the  hierarchical  aspects  of  pilot  mental  models. 

During  a  significant  part  of  their  mission  pilots  are  performing  essentially  a 
navigation  function.  They  are  at  high  altitude,  their  geographical  area  of 
awareness  is  relatively  large,  and  their  temporal  horizon  of  concern  is  rela¬ 
tively  far  into  the  future.  Since  they  are  flying  above  any  terrain  features 
that  might  be  hazardous,  their  model  is  essentially  two-dimensional;  terrain 
features  serve  mainly  as  navigational  cues,  and  their  main  concern  is  with  the 
combined  spatial/temporal  goals  of  following  a  route  that  will  avoid  threats 
and  reaching  waypoints  and  target  within  a  prescribed  window  of  time.  . Under 
these  circumstances,  pilots  rely  primarily  on  "God's-eye"  map-like  displays 
that  conform  to  this  high- altitude ,  two-dimensional,  large-area,  long- time- 
horizon  model. 

At  the  other  extreme,  some  of  their  time  is  spent  flying  low  to  avoid  radar 
detection,  maneuvering  at  low  altitudes  to  evade  missiles,  or  engaging  in  dog¬ 
fights  with  enemy  aircraft.  Under  these  conditions  their  geographical  area  of 
awareness  is  quite  small,  and  their  time  horizon  is  of  very  short  duration. 
They  are  intensely  concerned  about  potentially  hazardous  terrain  features ,  and 
their  model  is  very  much  a  three-dimensional  one.  Under  these  conditions 
pilots  place  a  heavy  reliance  on  direct  vision  outside  the  cockpit,  and  almost 
none  on  cockpit  displays.  Direct  vision  is  important  to  them  for  seeing  (1) 
missiles  they  are  trying  to  evade,  (2)  landmarks  on  the  bombing  run,  (3)  ter¬ 
rain  when  it  is  being  used  for  masking  at  low  altitudes,  (4)  terrain  that  may 
be  hazardous,  and  (5)  air-to-air  threats. 

Despite  the  large  difference  between  these  two  world  models,  pilots  can  ill 
afford  to  neglect  one  situation  completely  while  operating  in  the  other.  In 
the  interviews,  they  emphasized  the  importance  of  "thinking  ahead  of  the 
aircraft",  in  the  sense  of  mentally  preparing  to  respond  rapidly  to  changing 
situations.  Thus,  while  flying  at  high  altitude,  they  must  anticipate  the 
need  to  reduce  altitude  quickly  to  avoid  threat  tracking  radar  or  to  evade  a 
launched  missile,  and  mentally  rehearse  the  actions  they  would  take  if 
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necessary.  Similarly,  after  low  altitude  maneuvers,  they  must  anticipate  the 
need  to  make  up  time  (either  by  a  route  change  or  speed  change)  at  high  al¬ 
titude  in  order  to  achieve  their  desired  time  on  target.  One  of  the  pilots 
stated  that  shifting  from  one  point  of  view  to  the  other  took  time. 

Prototype  displays .  To  facilitate  the  pilot's  transition  between  the  high- 
altitude  and  the  low-altitude  condition,  a  series  of  displays  was  developed  to 
present  sequential  views  during  descent  and  ascent.  The  high- altitude  display 
was  always  shown  simultaneously  in  the  upper  right-hand  portion.  The  descend¬ 
ing  transition  displays  (Figures  A-18  to  A-22)  present  a  continuously  evolving 
change  from  a  high - alt itude ,  2-D,  wide  area  view  to  a  low-altitude,  3-D 
(perspective) ,  small-area  view.  During  this  transition,  the  threat  lethal 
contours  evolve  into  cones  shown  in  front  of  the  aircraft,  terrain  features 
are  shown  as  peaks  and  valleys  in  a  head-on  view,  and  the  originally  planned 
flight  path  becomes  foreshortened. 

During  the  ascending  series  (Figures  A-23  to  A-28),  the  reverse  sequence  is 
shown,  and  three  features  are  added:  (1)  a  recommended  route  for  mission 
recovery,  (2)  a  recommended  speed  for  recovery  of  time  on  target  (TOT),  and 
(3)  a  recommended  altitude  for  achieving  the  required  speed  with  economical 
use  of  fuel.  The  recommended  speed  and  altitude  are  also  shown  on  the  top- 
level  high-altitude  display  (Figure  A-28)  when  that  final  step  in  the  sequence 
is  shown. 

The  concept  allows  for  these  transition  displays  to  be  shown  either  before  a 
change  of  altitude,  to  give  the  pilot  a  preview,  or  during  the  change  to  help 
him  orient  to  the  new  conditions . 

In  the  sample  scenario',  the  descent  sequence  begins  after  the  pilot  views  a 
display  indicating  illumination  by  a  threat  radar  (Figure  A- 17) . 

Prototype  evaluation .  Pilots  responded  favorably  to  the  simultaneous  presen¬ 
tation  of  perspective  and  plan-view  displays : 


Simultaneous  3-D  and  2-D  displays 
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The  most  senior  pilot  suggested  that  perspective  displays  should  be  provided 
at  all  times  on  the  heads-up  display,  corresponding  to  what  the  pilot  would 
see  if  he  were  to  descend  to  low  altitude. 

One  pilot  thought  the  ability  to  preview  a  continuous  descent  sequence  was  a 
desirable  feature;  the  others,  however,  thought  the  transitional  displays  were 
not  needed.  Responses  were  generally  the  same  to  the  display  of  high  to  low 
altitude  transition  before  or  during  the  descent: 

High-to-low  altitude  transition 

Before  descent  266 

During  descent  276 

The  pilot  favoring  the  transition  display  thought  it  would  be  especially  valu¬ 
able  during  night  or  poor  visibility  conditions. 

None  of  the  pilots  saw  value  in  the  sequence  of  transition  displays  as  a 
preview  before  ascent  or  as  a  display  during  ascent: 

Low-to-high  altitude  transition 

Before  ascent  566 

During  ascent  566 

However,  here  again  the  value  of  simultaneous  plan- view  (high- altitude)  and 
perspectival  (low-altitude)  displays  was  noted. 

Responses  were  highly  favorable  to  the  display  of  a  recommended  route  for 
recovery  of  flight  plan: 

Recommended  recovery  route  213 

With  respect  to  the  display  of  recommended  speed  to  recover  time  on  target 
(TOT),  two  pilots  were  highly  favorable  and  one  was  neutral: 

Recommended  recovery  speed  114 
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The  neutral  pilot  thought  that  whether  one  was  ahead  or  behind  TOT  would  be 
obvious  from  a  display  of  projected  arrival  times  at  check  points,  and  that 
one  would  either  speed  up  or  dawdle,  as  necessary.  If  TOT  could  not  be 
achieved,  however,  he  would  want  to  be  informed. 

Pilots  were  strongly  favorable  toward  the  indication  of  being  illuminated  by  a 
threat : 

Threat  illumination  212 

One  pilot  pointed  out  that  some  way  of  de-cluttering,  or  distinguishing  among 
multiple  threats  in  terms  of  priority  and/or  level  of  confidence,  was  needed. 

Final  prototype  system.  The  pilots'  evaluations  confirmed  the  hypothesis  that 
transitions  between  different  cognitive  "worlds"  in  which  pilots  must  operate 
may  be  facilitated  by  simultaneous,  mutually  consistent  displays  representing 
those  worlds.  In  particular,  low-altitude,  three-dimensional  displays  prior 
to  and  during  descent  may  help  pilots  prepare  for  sudden  evasive  action  in 
terrain;  and  high-altitude  plan-view  displays  may  help  pilots  regain  a  large- 
scale  situation  understanding  prior  to  or  during  ascent.  Nevertheless,  pilots 
saw  little  value  in  sequential  displays  which  depicted  the  transition  between 
the  two  worlds.  Other  displays  that  supported  the  pilots'  ability  to  an¬ 
ticipate  new  circumstances  included  recommended  route  and  speed  for  recovery 
after  a  low- altitude  evasive  maneuver. 

No  changes  were  made  to  the  prototype  system  in  regard  to  these  displays . 
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4.0  CONCLUSIONS 


4 . 1  Summary  of  Findings  from  Phase  I 

Phase  I  was  successful,  both  on  theoretical  and  a  practical  level.  On  the  one 
hand,  some  new  insights  into  the  cognitive  foundations  of  pilot  performance 
were  obtained  from  a  review  and  analysis  of  the  cognitive  science  literature. 
On  the  other  hand,  implications  of  those  insights  for  pilot  displays  were  ex¬ 
tracted  and  successfully  tested.  Among  the  more  theoretical  conclusions  of 
the  Phase  I  work  are  the  following: 

o  Pilot  performance  can  be  represented  at  three  different  levels,  in¬ 
volving  skill-based,  stereotypical,  and  knowledge -based  performance. 

o  Stereotypical  performance  requires  a  characterization  in  terms  of 

highly  structured,  hierarchical  and  active  processes.  This  type  of 
knowledge  can  be  represented  in  a  framework  of  schemas  and  scripts. 

o  The  necessary  representational  properties  of  mental  models  can  be 

derived  from  their  function  of  generating  new  knowledge.  That  func¬ 
tion  implies  that  some  version  of  a  generate -and -test  process  is 
utilized  within  the  organism.  Such  an  internal  generate -and- test 
process,  in  turn,  implies  a  knowledge  representation  in  which  well- 
understood  components  are  "glued"  together  in  order  to  observe  their 
interaction.  Such  a  knowledge  representation  is,  in  fact,  a  type  of 
"analogical"  model,  in  which  the  components  correspond  one-to-one 
with  represented  objects  in  the  world,  and  in  which  conclusions  are 
"read  off"  from  the  model  itself,  without  the  benefit  of  previously 
existing  general  rules  or  knowledge. 

o  While  analogical  models  have  great  strengths  in  supporting  the 

ability  to  generate  new  knowledge,  they  are  unable  to  represent  in¬ 
determinacy  or  ambiguity  effectively. 
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o 


A  large  body  of  research  supports  the  finding  that  unaided  human 
problem  solving  is  characterized  by  biases  and  fallacies,  in  par¬ 
ticular  in  the  handling  of  uncertainty.  We  argue  that  many,  if  not 
all,  of  these  biases  and  fallacies  may  be  explained  by  the  human  use 
of  mental  models.  It  thus  follows  that  many  of  the  weaknesses  in 
human  reasoning  are  inextricably  intertwined  with  the  strengths  of 
human  reasoning,  i.e.,  the  ability  to  use  mental  models  to  generate 
new  knowledge . 

o  An  important  conclusion  is  that  there  is  a  requirement  for  a  design 
technology  that  both  accommodates  natural  human  knowledge  structures 
and  at  the  same  time  helps  users  avoid  the  inherent  pitfalls  in 
those  structures. 

o  A  design  methodology  of  this  type,  called  personalized  and  prescrip¬ 
tive  decision  support,  is  proposed.  This  methodology  involves 
modeling  both  user  cognitive  processes  and  representations,  on  the 
one  hand,  and  normatively  correct  solutions  to  the  problem  on  the 
other  hand.  These  models  are  compared,  and  the  potential  strengths 
and  weaknesses  of  the  user-preferred  approach  are  determined.  Dis¬ 
plays  are  designed  which  preserve  the  strengths  of  the  user- 
preferred  approach,  i.e.,  which  do  not  require  users  to  adopt  radi¬ 
cally  different  "normative"  techniques  of  problem  solving.  At  the 
same  time,  however,  these  displays  guard  against  specifically  iden¬ 
tified  shortcomings  in  the  user  approach.  Therefore,  the  end  result 
should  be  performance  which  satisfies  the  constraints  of  the  norma¬ 
tive  model,  while  at  the  same  time  more  effectively  communicating 
with  the  user  and  eliciting  on-the-spot  user  knowledge. 

o  Personalized  and  prescriptive  decision  support  may  take  either  of 
two  forms.  In  one  case,  adaptation  to  the  user's  mode  of  problem 
solving  is  primary.  The  display  facilitates  the  user's  preferred 
approach,  but  monitors  his  performance  and  prompts  him  when  his  own 
approach  is  likely  to  lead  to  serious  errors.  In  the  other  ap¬ 
proach,  adaptation  to  the  situation  via  the  normative  model  is 
primary.  However,  the  computer  monitors  its  own  performance,  and  in 
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cases  where  it  detects  weaknesses  or  conflict  in  its  own  line  of 
reasoning,  where  the  user  is  judged  to  have  potentially  valuable  in¬ 
formation,  and  where  the  user's  workload  is  at  an  acceptable  level, 
the  aid  prompts  the  user  for  a  contribution. 

Pilot  mental  models  were  elicited  in  structured  interviews  in  which  pilots 
answered  questions  about  the  objects  and  parameters  of  concern  to  them  in  a 
series  of  hypothetical  situations.  Displays  were  then  designed  to  conform  to 
the  constraints  imposed  by  the  theory  of  mental  models .  These  displays  were 
implemented  in  a  demonstration  computerized  system  which  was  then  reviewed  and 
evaluated  by  pilots .  Preliminary  conclusions  and  candidate  display  concepts 
include  the  following: 

o  In  cases  of  uncertainty  about  threat  location  or  threat  identity, 
pilots  prefer  displays  of  specific  possible  situations  (e.g.,  that 
assume  a  particular  threat  location  or  type)  to  displays  that  prob¬ 
abilistically  aggregate  over  the  alternatives.  Aggregated  displays 
correspond  to  no  actualizable  situation,  and  thus  may  disrupt  the 
pilot's  effort  to  "stay  ahead  of  the  airplane"  with  mental  models 
that  concretely  anticipate  future  circumstances. 

o  In  cases  of  conflicting  evidence,  pilots  prefer  situation  displays 
which  represent  "worst  case"  as  opposed  to  "best  case"  assumptions 
about  threat  location  or  identity,  i.e.,  displays  that  depict  the 
possibility  with  the  greatest  potential  impact  on  the  mission. 

o  Nevertheless,  pilots  found  the  option  of  viewing  a  best  case 

scenario  highly  acceptable.  Two  of  the  three  pilots  preferred  the 
ability  to  compare  worst  and  best  case  scenarios  for  themselves, 
rather  than  viewing  an  aggregated  "average"  scenario.  Such  options 
provide  a  counterbalance  to  the  pilot's  tendency  to  focus  ex¬ 
clusively  on  a  single  possibility. 

o  In  cases  of  incomplete  evidence  about  a  new  threat,  pilots  appeared 
to  adopt  a  modified  "best  case"  assumption,  especially  if  taking  ac¬ 
tion  in  regard  to  the  new  threat  would  itself  incur  risk. 
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Recommended  routes  for  avoiding  an  unanticipated  ground  threat  were 
strongly  welcomed  by  pilots.  As  a  prescriptive  counterbalance  to 
the  tendency  to  focus  on  a  single  concrete  assumption  (worst  case) , 
such  recommendations  could  be  accompanied  by  prompts  when  the  best 
response  on  a  worst  case  assumption  would  be  significantly  inferior 
to  a  strategy  of  "hedging"  against  uncertainty. 

o  To  the  extent  that  pilots  explicitly  deal  with  uncertainty,  they 

utilize  mental  models  centered  on  potential  sources  of  data.  Thus, 
pilots  attempt  to  correlate  incoming  sensor  reports  and  radio  mes¬ 
sages  with  prior  intelligence  about  expected  threats  along  a  planned 
route;  concern  is  aroused  when  these  sources  are  in  conflict.  An 
effective  mental  model  display,  therefore,  directly  depicts  each  of 
the  potential  sources  of  data  regarding  a  threat  (prior  intel¬ 
ligence,  own  aircraft  sensors,  AWACS ,  etc.)  as  an  icon.  The  color 
of  the  icon  directly  encodes  the  impact  of  that  source  of  data 
(green  -  supports  best  case;  red  =  supports  worst  case);  while  the 
intensity  of  the  icon  directly  encodes  the  credibility  of  the  source 
of  data.  The  pilot  can  thus  tell  at  a  glance  the  extent  and  nature 
of  any  conflict  (if  all  icons  are  green  or  all  are  red,  there  is  no 
conflict;  a  mix  of  red  and  green  means  uncertainty). 

o  Pilots  felt  comfortable  with  the  idea  of  providing  their  own  inputs 
within  this  framework,  by  reducing  the  credibility  of  one  or  more 
data  sources  either  directly  or  by  responding  to  queries  (e.g., 
about  presence  of  countermeasures,  visibility,  etc.). 

o  Pilots  sometimes  need  to  think  simultaneously  about  two  "worlds"- - 

e.g.,  to  plan  for  a  possible  sudden  descent  while  flying  at  high  al¬ 
titude.  To  facilitate  this  process,  pilots  strongly  favored  the 
simultaneous  presentation  of  two  mutually  consistent  displays 
depicting  a  large-area,  two  dimensional  long-time  horizon  model  and 
a  narrow- area,  three  dimensional,  short-time  horizon  model. 
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4 . 2  Future  Directions 


The  principal  lesson  from  Phase  I  of  this  research,  we  feel,  is  that  a  mix  of 
cognitive  science  theory  and  empirical  testing  can  lead  to  rapid  progress  in 
the  development  of  cognitively  compatible  displays.  The  theory  provides  (a)  a 
framework  for  understanding  how  pilots  represent  knowledge  and  how  such  repre¬ 
sentations  contribute  to  effective  performance;  (b)  a  set  of  methods  for 
designing  displays  that  conform  with  the  constraints  of  pilot  internal 
representations;  and  (c)  interactive  techniques  for  counteracting  the  cogni¬ 
tive  biases  with  which  those  representations  are  associated.  Pilots  them¬ 
selves  play  a  critical  role  in  this  process.  Structured  interviews  provide  an 
initial  test  of  the  hypotheses  generated  by  cognitive  theory  and  (if  the 
hypotheses  are  confirmed)  help  us  flesh  out  the  details  of  the  pilot' s_ actual 
internal  models.  Review  by  pilots  of  preliminary  prototype  system  displays 
provides  another  test  of  the  hypotheses  and  further  refinement  of  the  display 
concepts . 

Future  research  will  continue  the  application  of  the  cognitive  design 
methodology  described  in  this  report  to  a  wider  range  of  pilot  in-flight  deci¬ 
sion  making  tasks;  will  incorporate  the  resulting  displays  and  interactive 
principles  into  a  prototype  real-time  pilot  aid;  and  will  hopefully  lead  to 
the  development  of  more  general  guidelines  and  methods  for  the  design  of  cog¬ 
nitively  compatible  interactive  displays. 

Among  the  areas  for  further  research  are  the  following: 

o  Routine  performance.  Research  concerning  scripts  and  schemas  have 
as  yet  unexplored  implications  for  pilot  display  design.  To  what 
extent  should  the  information  presented  to  pilots  and  the  modes  of 
interaction  between  pilot  and  the  aide  vary  as  a  function  of  current 
goals  and  activities?  For  example,  the  display  of  threat  danger  may 
vary  significantly  during  planning,  on  the  ingress,  during  the  at¬ 
tack,  and  on  the  egress.  Similarly,  interactive  methods  for  gener¬ 
ating  and  evaluating  new  routes  and  tactics  may  also  vary  as  a  func¬ 
tion  of  where  in  the  mission  these  activities  occur.  These  displays 
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may  also  vary  as  a  function  or  specific  sub-goals  and  conditions, 
such  as  altitude,  current  threat  density,  and  fuel  status. 

o  Problem  solving  performance .  Another  area  of  application  involves 
decision  making  tasks  which  the  pilot  faces  from  time  to  time,  e.g., 
in  determining  courses  of  action  against  unexpected  threats  or  ex¬ 
plaining  unexpected  events.  In  these  contexts,  displays  must  be 
provided  which  are  compatible  with  the  user's  cognitive  style,  but 
which  at  the  same  time  provide  prompts  or  other  display  features 
which  guard  against  decision  making  biases.  For  example,  in  cases 
of  uncertainty  due  to  conflicting  evidence,  we  have  recommended  that 
pilots  be  provided  with  worst  case  displays  along  with  the  option  of 
viewing  displays  that  represent  other  possibilities.  An  additional 
protective  device  against  potential  baises  might  be  provided  in  the 
form  of  prompts  which  warn  pilots  when  a  course  of  action  based  on 
the  worst  case  assumption  may  be  significantly  inferior  to  actions 
which  exploit  other  possibilities  or  which  hedge  against  uncer¬ 
tainty.  Another  promising  area  of  application  involves  choice  among 
options  which  vary  on  multiple  attributes.  For  example,  after  an 
evasive  maneuver  the  pilot  may  be  unable  to  arrive  at  the  target  by 
the  designated  time  with  the  preplanned  course  and  speed.  The 
process  of  replanning  involves  balancing  increased  risk  to  own 
aircraft  against  the  importance  of  the  target,  as  well  as  other  fac¬ 
tors  such  as  dependence  of  other  aircraft  on  performance  of  the  mis¬ 
sion  and/or  the  possible  substitution  of  other  aircraft  in  the  mis¬ 
sion.  Displays  are  needed  which  help  pilots  organize  and  evaluate 
these  factors ,  and  which  guard  against  the  danger  of  disregarding 
significant  information. 

o  Human  computer  task  allocation .  Traditionally,  task  allocation  in 

human-machine  systems  has  been  course -grained  and  inflexible:  tasks 
are  rigidly  assigned  to  the  computer  or  to  the  user  according  to  the 
purported  strengths  of  each.  The  display  methodology  described  in 
this  report  opens  the  way  to  a  more  flexible  and  dynamic  approach, 
in  which  the  balance  of  initiative  between  human  and  computer  shifts 
back  and  forth  as  a  function  of  workload,  relative  expertise,  and 
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user  preferences.  The  key  concept  is  that  in  all  task  allocation 
modes,  user  and  computer  complementarity  is  maximally  exploited. 

Thus  under  circumstances  when  problem-solving  is  under  the  user's 
initiative,  the  computer  monitors  the  user's  decision  making  be¬ 
havior  and  provides  prompts  when  that  behavior  significantly  vio¬ 
lates  normative  constraints.  Under  circumstances  when  problem¬ 
solving  is  primarily  under  computer  initiative,  the  computer 
monitors  its  own  performance  for  incompleteness  of  evidence  or  con¬ 
flict  among  data  sources,  and  prompts  the  user  when  the  user  is 
likely  to  be  able  to  make  a  significant  contribution. 

Pilot  interaction  with  in-flight  intelligent  systems  remains  both  a  highly  ur¬ 
gent  and  a  highly  promising  area  for  the  application  of  cognitive  science  dis¬ 
play  technology. 
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APPENDIX 


PROTOTYPE  SYSTEM  DISPLAYS 


This  appendix  contains  a  description  of  the  displays  included  in  the  prototype 
system,  presented  in  the  order  of  the  sample  scenario.  The  user  of  the  sys¬ 
tem,  however,  would  not  necessarily  see  these  exact  displays  in  this  exact  or¬ 
der,  since  in  some  cases  what  he  would  see  would  depend  on  his  own  choices. 
While  the  displays  presented  here  do  not  include  all  those  which  were 
developed  for  the  prototype  system,  they  provide  a  representative  sampling  of 
different  user  menu  choices  and  different  user  actions. 


A-l 


Figure  A-l 


The  scenario  begins  with  own  aircraft  (blue  aircraft  symbol)  having  crossed 
the  FEBA  (yellow  dotted  line)  on  a  planned  route  (solid  blue  line)  to  a  ground 
strike  target  (yellow  "T") .  Ground  threats  are  represented  by  generic  symbols 
for  surface  to  air  missiles,  anti- air  artillery,  and  radar.  Different  shades 
of  red  indicate  different  levels  of  threat  to  the  aircraft. 


Figure  A-2 


The  aircraft  has  now  received  information,  via  an  electronic  data  link  from  an 
AWACS,  indicating  a  possible  threat  along  its  planned  route.  Since  this  data 
is  regarded  by  the  system's  inference  mechanism  as  insufficiently  reliable  on 
its  own  to  establish  the  existence  of  the  threat,  and  since  it  has  not  been 
confirmed  by  other  sources,  the  existence  of  the  threat  is  not  established, 
and  a  modified  best  case  situation  display  is  presented  to  the  user.  This 
consists  of  a  yellow  outline  around  the  region  where  the  unconfirmed  threat 
might  exist,  with  a  question  mark  indicating  the  uncertainty.  In  addition, 
icons  to  the  left  of  the  display  graphically  indicate  the  status  of  various 
data  sources  in  regard  to  this  threat.  Each  icon  stands  for  a  data  source: 
from  top  to  bottom,  the  folder  represents  pre-briefed  intelligence,  the 
aircraft  stands  for  on-board  sensors  and  pilot  visual  observation,  and  the 
lightning  bolt  stands  for  electronic  data  link  messages  from  friendly  air  or 
ground  stations  (e.g.,  through  JTIDS) .  Red  icons  support  the  worst  case 
assumption  (existence  of  the  new  threat) ;  green  icons  support  the  best  case 
assumption  (no  new  threat) ;  and  blank  icons  reflect  inconclusive  or  unreliable 
data.  Thus,  the  pilot  can  see  at  a  glance  both  how  much  support  there  is  for 
a  particular  possibility  and  where  it  is  coming  from.  The  red  lightning  bolt 
indicates  that  the  data  link  source  (i.e.,  the  AWACS)  supports  the  existence 
of  the  threat;  however,  the  blank  icons  indicate  that  pre-briefed  intelligence 
and  on-board  sensors  respectively  have  provided  no  reliable  information  on  the 
presence  or  absence  of  this  threat. 


Figure  A-3 


If  he  wishes  to,  the  pilot  may  request  a  worst  case  display.  This  indicates 
in  more  detail  the  lethality  contour  of  the  threat,  assuming  that  the  threat 
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Figure  A- 4 


At  a  somewhat  later  point  in  time,  confirmation  for  the  existence  of  a  new 
threat  is  received  from  on-board  sensors.  As  a  result,  the  inference 
mechanism  establishes  the  existence  of  the  threat,  and  the  displayed  situation 
now  corresponds  to  the  worst  case  possibility.  Reflecting  the  new  informa¬ 
tion,  the  data  source  icon  indicating  own  aircraft  sensors  is  now  displayed  in 
red.  The  pilot  can  again  see  at  a  glance,  by  looking  at  the  icons,  how  much 
support  is  present  for  a  particular  possibility.  The  yellow  contours  in  the 
situation  display  indicate  regions  where  danger  to  own  aircraft  has  increased, 
by  a  specific  percentage,  on  account  of  the  new  information.  An  auditory 
alert  accompanies  this  display. 


Figure  A- 5 


The  pilot  may  request  that  the  system  provide  a  recommended  route  to  avoid  the 
new  threat.  The  recommended  route  revision  is  shown  in  purple. 
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Figure  A- 6 


The  pilot  has  indicated  his  acceptance  of  the  new  route.  As  a  result,  the 
original  route  plan  is  revised.  The  new  threat  is  now  shown,  like  other 
threats,  in  red  (as  opposed  to  the  yellow  contours  whose  purpose  was  to  indi¬ 
cate  new  information) . 
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Figure  A- 7 


At  this  time  the  aircraft  receives  another  electronic  data  link  message  from 
the  AWACS  regarding  a  second  possible  unexpected  threat.  The  available 
evidence  in  this  situation  is  consistent  with  two  possibilities:  there  is  an 
additional  unexpected  surface-to-air  missile  site  along  the  planned  route  of 
the  aircraft,  as  shown  in  this  figure;  or  a  previously  identified  surface-to- 
air  missile  site  has  either  moved  somewhat  to  the  northeast  or  was  previously 
mislocalized.  The  AWACS  information  supports  the  first  possibility,  while 
pre-briefed  intelligence  supports  the  second  possibility  (i.e.,  there  is  sub¬ 
stantial  confidence  that  no  new  sites  have  been  introduced  into  the  area) . 

Own  aircraft  sensor  information  is  consistent  with  both  possibilities.  In  ac¬ 
cordance  with  the  pilot's  mental  model  of  this  situation,  he  is  automatically 
provided  with  a  worst  case  display  (i.e.,  Figure  A-7).  The  icons  on  the  left 
of  the  screen  graphically  indicate  the  directions  in  which  each  data  source  is 
pointing.  Thus,  the  pilot  can  tell  at  a  glance  whether  data  sources  are  in 
agreement  (all  green  or  all  red)  or  are  in  conflict,  as  in  this  case.  In  ad¬ 
dition,  an  explicit  "CONFLICT"  indicator  is  provided  above  the  icons. 


A- 14 


A- 15 


Figure  A- 8 


If  he  wishes,  the  pilot  may  examine  best  case  possibilities  as  well.  In  this 
display,  the  new  information  from  the  AWACS  is  interpreted  on  the  assumption 
that  it  represents  a  moved  or  mislocalized,  but  previously  known,  threat. 
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Figure  A- 9 


The  pilot  also  has  the  option  of  examining  an  average,  or  probabilistically 
aggregated,  display.  In  this  display,  the  danger  at  any  given  point  is  the 
weighted  average  of  the  danger  on  the  worst  case  possibility  and  on  the  best 
case  probability,  with  weights  corresponding  to  the  probabilities  of  those  two 
situations. 
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Figure  A- 10 


The  pilot  may  request  a  recommended  route  revision  based  on  the  new  informa¬ 
tion.  Such  a  revision  may  be  requested  in  the  context  of  the  worst  case  dis¬ 
play,  the  best  case  display,  or  the  average  display.  This  figure  shows  the 
recommended  route  on  the  worst  case  assumption. 
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Figure  A- 11 


This  figure  shows  the  recommended  route  in  the  context  of  the  probabil¬ 
istically  aggregated  display. 
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Figure  A- 12 


Here  the  pilot  has  indicated  acceptance  of  the  route  based  on  the  probabil¬ 
istically  averaged  display.  The  recommended  revision  is  incorporated  into  the 
previously  planned  route,  and  the  yellow  contours  (indicating  unexpected 
information)  are  replaced  by  the  standard  red  contours.  The  question  mark 
remains  to  indicate  continuing  uncertainty  regarding  the  existence  and/or 
location  of  the  threat. 
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Figure  A- 13 


In  the  case  of  disagreement  among  sources  of  data,  the  system  provides  the  op¬ 
portunity  to  resolve  the  conflict  by  discounting  one  or  more  of  the  sources. 
The  inference  mechanism  automatically  examines  potential  causes  of  the  con¬ 
flict,  i.e.,  assumptions  upon  which  one  or  more  of  the  conflicting  data 
sources  depend  for  their  credibility.  For  example,  radar  data  may  be  affected 
by  ground  reflectance,  weather,  or  electronic  countermeasures.  If  the  system 
can  automatically  resolve  the  conflict,  it  does  so  (by  additional  data  collec¬ 
tion  or  data  analysis).  If  it  cannot,  and  if  the  conflict  is  significant  for 
mission  success  or  aircraft  safety,  the  system  queries  the  user  regarding  fac¬ 
tors  that  would  potentially  discredit  one  or  more  of  the  sources.  Thus,  in 
Figure  A- 13  the  system  has  asked  the  pilot  whether  the  presence  of  ECM,  which 
would  invalidate  the  AWACS  evidence,  is  likely.  The  pilot  is  free  to  respond 
to  this  query,  ignore  it,  or  indicate  "no  information."  If  he  indicates  the 
latter,  the  system  may  produce  an  additional  query. 
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Figure  A- 14- 


In  this  figure,  the  pilot  has  responded  to  the  query  by  indicating  that  ECM 
affecting  the  AWACS  is  probable;  the  AWACS  evidence  has  been  discounted,  as 
shown  by  the  blank  lightning  bolt  icon;  and  the  conflict  has  been  resolved  by 
the  inference  mechanism  in  favor  of  the  best  case  possibility.  The  pilot 
could  have  indicated  his  lack  of  confidence  in  a  data  source  more  directly 
simply  by  pointing  and  clicking  at  the  icon  representing  that  data  source. 
When  he  does  so,  the  data  source  is  discounted  (i.e.,  the  icon  becomes  blank), 
and  the  conflict  is  resolved  in  the  appropriate  direction. 
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Figure  A- 15 


After  resolution  of  the  conflict,  the  pilot  requests  a  recommended  route  in 
the  context  of  the  best  case  possibility.  The  recommended  route  is  shown  in 
purple . 
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Figure  A- 16 


The  user  has  indicated  his  acceptance  of  the  recommended  route. 
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Figure  A- 17 


Later  in  the  scenario,  on  the  way  to  the  target,  onboard  sensors  indicate  that 
the  aircraft  has  been  illuminated  by  a  surface-to-air  threat.  The  yellow  ar¬ 
row  from  the  threat  to  the  aircraft  represents  the  increased  danger  in  this 
situation.  An  auditory  alert  is  also  provided. 
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Figure  A- 18 


As  a  result  of  the  threat  illumination,  the  pilot  decides  to  descend  to  lower 
altitude  to  exploit  terrain.  Such  a  descent  involves  a  rapid  alteration  in 
the  pilot's  viewpoint:  from  a  large-scale,  two-dimensional  plan-view  to  a 
narrow,  three-dimensional  perspectival  view.  To  facilitate  this  transition, 
the  system  presents  a  sequence  of  views  which  anticipates  what  the  pilot  will 
see  on  his  descent.  Figure  A- 18  shows  the  aircraft  in  the  initial  portion  of 
the  descent.  A  plan-view  situation  display  is  shown  simultaneously  in  the  up¬ 
per  right. 
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Figure  A- 19 


The  sequence  of  displays  corresponding  to  the  descent  continues .  As  the 
"point  of  view"  of  the  display  descends,  it  also  begins  to  look  ahead  rather 
than  down.  As  this  happens,  features  of  the  display  evolve  in  a  continuous 
manner:  i.e.,  threat  lethal  contours  become  cones  shown  in  front  of  the 
aircraft,  terrain  features  are  shown  as  peaks  and  valleys,  and  the  planned 
aircraft  route  becomes  foreshortened. 
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Figure  A-20 


The  descent  sequence  continues. 
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Figure  A-21 


The  descent  sequence  continues. 
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Figure  A-22 


This  is  the  final  display  in  the  descent  sequence,  showing  the  aircraft  at  its 
lowest  planned  altitude. 
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Figure  A-23 


During  or  prior  to  the  ascent  back  to  standard  altitude,  the  system  provides  a 
corresponding  sequence  of  displays  which  shows  the  aircraft  on  the  ascent. 
Again,  display  features  evolve  continuously  during  the  transition,  and  the 
large-scale  plan-view  situation  display  continues  to  be  shown  simultaneously. 
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Figure  A- 2  A 


After  the  ascent,  the  pilot's  objective  is  to  recover  his  original  flight  plan 
to  the  target.  Thus,  this  display  in  the  ascent  sequence  provides  a  recom¬ 
mended  route,  speed,  and  altitude  for  recovery. 
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Figure  A-25 


This  is  the  next  display  in  the  ascent  sequence.  The  "point  of  view"  of  the 
display  begins  to  look  down  (as  opposed  to  forward)  as  the  aircraft  increases 
in  altitude. 
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Figure  A-26 


This  is  the  next  display  in  the  ascent  sequence. 
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Figure  A-27 


This  is  the  next  display  in  the  ascent  sequence. 
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Figure  A- 28 


The  plan-view  situation  display  is  resumed,  showing  a  recommended  speed  and 
altitude  to  reach  the  target  by  the  designated  time  at  minimum  risk. 
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Figure  A-29 


A  short  while  later,  the  pilot  is  again  illuminated  by  a  surface-to-air 
threat.  The  "X"  over  the  yellow  arrow  indicates  that  on-board  electronic 
countermeasures  have  effectively  negated  this  threat. 


Figure  A- 30 


The  pilot  has  successfully  struck  the  target  and  is  entering  the  egress  por¬ 
tion  of  the  route . 
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Figure  A-31 


At  this  time,  the  pilot  again  receives  unexpected  information,  this  time  per¬ 
taining  to  the  classification  of  a  threat.  On-board  EW  equipment  indicates 
that  a  surface-to-air  site  near  the  planned  egress  route  may  be  an  SA-4  as  op¬ 
posed  to  an  anticipated  SA-2.  (Note  that  these  displays  use  fictional 
parameters  for  threat  capabilities.  The  SA-4  is  thus  regarded  as  more  capable 
than  the  SA-2.)  Figure  A-31  shows  the  worst  case  situation  which  is  automati¬ 
cally  provided  to  the  pilot,  i.e.,  classification  as  an  SA-4.  Yellow  contours 
indicate  areas  where  the  increased  danger  to  the  aircraft  due  to  new  informa¬ 
tion  has  exceeded  a  certain  threshold.  The  icons  on  the  left  indicate  what 
various  data  sources  are  saying  about  threat  classification:  i.e.,  green  in¬ 
dicates  support  for  classification  as  an  SA-2  (best  case),  and  red  indicates 
support  for  classification  as  an  SA-4  (worst  case) .  The  mix  of  red  and  green 
in  the  icon  display  thus  indicates  to  the  pilot  at  a  glance  that  significant 
conflict  exists  regarding  this  threat.  An  explicit  "CONFLICT"  indicator  is 
also  provided,  above  the  icons. 
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Figure  A-32 


If  he  wishes,  the  pilot  may  also  examine  the  situation  under  best  case  assump¬ 
tion,  i.e.,  assuming  that  the  threat  is  an  SA-2.  Since  this  assumption  cor¬ 
responds  to  the  prior  expectation,  no  yellow  contours  are  shown  on  this  dis¬ 
play. 
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Figure  A- 33 


The  pilot  also  has  the  option  of  viewing  a  probalistically  aggregated,  or 
average,  display.  In  this  display  the  danger  at  any  given  point  is  a  weighted 
average  of  the  dangers  on  each  of  the  two  possibilities,  where  the  weights 
correspond  to  their  probabilities. 
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Figure  A-34 


The  pilot  has  requested  a  recommended  route  to  accommodate  the  new  informa¬ 
tion,  based  on  the  probabilistically  aggregated  display.  The  recommended 
route  is  shown  in  purple . 
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Figure  A-35 


The  pilot  has  accepted  the  recommended  route.  As  a  result,  the  route  revision 
has  been  incorporated  into  the  preplanned  route,  yellow  contours  have 
disappeared,  and  uncertainty  continues  to  be  acknowledged  by  the  presence  of 
the  question  mark. 
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Figure  A- 3 6 


The  pilot  continues  the  egress  towards  the  FEBA. 
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